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ABSTRACT 

At the eighth Western Regional Conference on Testing 
Problems, the following speeches were given: (1) "Background for 
Recent Curriculum Developments" by Ralph W« Tyler; (2) "Recent 
Curriculum Developments in Mathematics" by Robert E. K# Rourke; (3) 
"Recent Curriculum Developments in Science" by Kenneth E. Davis; (^) 
"Changes in Measurement as a Result of Curriculum Developments" by S. 
A# Kendrick; and (5) "Judging the Usefulness of a Test in Curriculum 
Measurement" by Robert L. Ebel. A list of conference participants 
concludes the report. (KM) 
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The Eighth Annual 
WESTERN REGIONAL CONFERENCE ON TESTING PROBLEMS 



The morning meeting of the Eighth Annual Western Regional Conference on Testing 
Problems, held in the Blossom Xloom, Hollywood Roosevelt Hotel, Los Angeles, 
California, convened at 9:30 a.m.. Dr. Arthxir P. Coladarci, General Chairmaji, 
presiding. Dr. John S. Helmick, Director of the Los Angeles Office of Educational 
Testing Service, welcomed the conferees. 

DR. JOHN HELMICK: I am beginning to feel like an old timer at this. This is my 
third opportunity to introduce an Annual Western Regional Conference on Testing 
Problems. This is the eighth such conference, however, so I'm still really a rel- 
ative newcomer. 

We appreciate your continued support. Every year we have had not only a bigger 
conference, but I think each succeeding year we have had a better conference. We 
appreciate your suggestions as to how these axmxial meetings may be improved so 
that they will be more useful to you. We frequently get contradictory suggestions, 
so it^s not possible to please everybody; but we will endeavor to do the best we can. 

Since this is the last time I will be before you today, I want to take this opportunity 
tp thank the participatnts in advance, not that I feel it wouldn't be appropriate to 
t^hank them afterwards, but I'm not going to be arotind to do it. The cooperation 
they provide is absolutely essential, of course, to a conference such as this. This 
is particularly so since their contribution to measurement is essentially their only 
reward. We appreciate very much their willingness to help. 

Most of the members of the staff of the Los Angeles office of the Educational Test- 
ing Service have been involved in some way in the preparation and plaiining for this 
conference. My own role has been a most minimal one. Primarily, it consists of 
having the opportxmity to thank all these people for their Help, particularly Robert 
Laanbert, who is in charge of our Advisory Services, ajid as such has had general 
responsibility for the conference. I also wish to thank Mrs. Mary Owen, who has 
handled the many arduous details of the necessary arrangements. Without them, it 
is clear we would have had no conference. 

My only other task is to introduce the General Chairman co you. As is so often the 
case, most of you do not need this introduction. Nevertheless, I have some infor- 
mation; and before I sit down, I'm going to tell you a little bit about him for the sake 
of those few who may not be familiar with him. He is Professor of Education and 
Psychology at Stanford University. His origins were in Connecticut. He went to 
school there, obtained his Ph.D. at Yale University, with some additional graduate 
studies at New York University and Johns Hopkins. He taught in the Danbury Public 
Schools, at the University of Connecticut, and New Haven State Teachers College. 
He began the western migration with a stopover at the University of Indiatna. As I 
have indicated, he has ended up, as so many good people do, at Stanford. 
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He has been a consulting editor for the Dryden Press and is now serving in that 
capacity for the John Wiley Press. Among his unique activities has been a year as 
a senior Fxillbright research scholar in Japan. Apparently, be is in the process of 
becoming a Japanese expert, for in September, he will be a delegate to the Inter- 
national Conference on Educationa. Research in Tokyo. As many of you' know, he 
has authored mauiy books and articles in the field of educational psychology, edu- 
cational research, and educational criticism. I am very happy now to turn the rest 
of the meeting over to Dr. Arthur Coladarci of Stanford University, who is General 
Chairman of the conference. 



GENERAL CHAIRMAN ARTHUR P, COLADARCI; Thank you. Dr. Helmick. A 
few years ago, the people then directing the Western region for ETS circulated a 
questionnaire among participants at one of their conferences, asking for criticisms 
of it. I find it very difficult to resist an invitation of this sort; and since negative 
criticism is easy to give, I did precisely that. I don't remember what I said, ex- 
cept it was negative. One of the consequences of that was the invitation to act as 
Chairman toddy. This is a form of co-optation. 

Fortxinately, this conference appears to be the best that the ETS people have ever 
planned. I say this not because of the chairmanship, but because of the people who 
are going to speak, as well as the kind of people who are here in the audience - a 
very distinguished looking group, of whom I know none. I see some faithful grad- 
uate students of mine and some even more faithful persons yet in training, the 
most faithful kind of all. It is on these bases that I feel we are going to have a very 
comfortable day. 

We have a tight schedule because at least one of our speakers, Mr. Rourke, insists 
on using all the time assigned to him. Therefore, I have really no time in which to 
say anything important. 

Let me briefly note that the general plan for today is to have three speakers on the 
general curricultim patterns that are now emerging ajid problems involved in them. 
This afternoon, Mr. Kendrick and Mr. Ebel will speak to the question of what im- 
plications flow for the psychometric task from the present status of affairs in cur- 
riculum. This means education generally. I don't think there is any intention to 
focus on any one level. It happens, largely by accident, that most of the speakers 
will be representing the secondary and higher education fields. Their intent, how- 
ever, is to speak to the general problem, not to the specific educational level they 
represent. 

After eack speaker, you will have an opportxmity to get rid of your own aggressions, 
hostilities., and anxieties in the form of questions. These are earnestly solicited. 
You are warned to do only one thing, that is to identify yourself very clearly by name 
and origin so far as it represents your current address. You may resist any evalu- 
ations of it if you wish. You must say this clearly, because the proceedings will 
identify you by najne. It is embarrassing to say something brilliant and find it at- 
tributed to somebody else. The converse is comforting, but you can't predict it. 



ERLC 



I have K couple of generalizations which I think are relevant to the theme of today's 
session as well as to the content of each of the five papers and the remarks you will 
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hear. We have often told ourselves (and I think we mean i'*,) that developments in 
testing have far outreached our ability to use them intelligently. I think this is 
true. ^Ve continue to be \inintelligent with the procedures we have and very fre- 
quently complain that the procedures are not up to the task in which we are en- 
gaged. What is distressing, however, is that very frequently, tasks in which we 
are engaged are without rationale; so psychometric questions can't really be 
raised intelligently. It is for this reason that I am particularly interested in 
hearing the first three paper u, which are addressed to the question of what cur- 
riculum situation confronts v.s, from which one presumably raises psychometric 
questions. 

There are, in education, many viewpoints regarding tests. If somebody rotates 
an axis through them, he would come out, I suppose, nauseous* There is, for 
instance, a large group, which I identify as the sentimentalists, who feel that if 
you administer a test, you "donH really find out about the child". This group, 
which is w(j11 entrenched in high places sometimes^^ makes statements such as 
these. These are actual quotations. If you are interested in their source, I'll be 
glad to give them to you privately, since no one of them comes from my own in- 
stitution* Recently, we have had the very interesting, even though somewhat fat- 
uous, statement: "We need to get more teachers who are interested in evaluation 
rather than measurement," This^ semantic massaging continues. A distinction is 
made between two kinds of operations, one of which bothers the author. This is 
called "measurement". I am not sXive what is involved in it. Evaluation, however, 
a more favorable term, is ambiguous enough to permit euiything intelligent to be 
said \inder it. 

Another and more recent statement interests me even more because I think it is 
particularly naive. A very prominent perj^on made the statement that he does not 
want to use tests when he is working with pupils because he "wants nothing to inter- 
vene betv/een himself and the pupil". This is a rather interesting statement and 
also is an argument against a lot of other things, if you think about it very carefully. 
This person thinks "tests are biasing". Apparently there exists an osmotic proc- 
ess whereby an instructor immediately perceives all the manifestations of the be- 
havior with which he is coifrcnt ed, and tests would get in the way. Statements like 
this represent what I call sentimentalist factors among educators. There may be 
some sentimentalists in this room. In that case, I wonH apologize: I'll merely ac- 
cord you the courtesy of not identifying you by name. 

At the other extreme, there is the kind of person who just loves to take correlation 
matrices and rotate axes through them. He doesn't care what comes out as long as 
the decimal point is in the correct place. These people, I think, also are not being 
very helpful. 

Well, no one of these two groups is represented up front today. Probably some 
are in the audience, and you may be quite irritated with some of the comments to be 
made, because all five people are going to speak rather frankly - and, I think, very 
intelligently* I read the papers, and I agree with every single one of them. There 
is one slight conflict of which Dr. Tyler and Mr. Rourke are not yet aware. And I 
think Mr* Ebel will be shocked to hear one of the things that Dr. Tyler will say. He 
doesn't know about it, but he has already written his response. He's going to have to 



do some rapid modification in the course of the morning. These people are going 
to strongly and forcefully speak on the assximption that you are willing to listen 
and perhaps be persuaded* 

* 

At the end of the afternoon, I have some other comments which I hope will be rel- 
evant. I have already taken some of the time^from each of the speakers. I will 
introduce them as we go along. 

We are going to start off this morning with Dr. Tyler, whom you all know, I*d-be 
very surprised if people in education didn^t know about him. When someone asks, 
"How are you?'', you immediately respond, "Fine.'" (Sometimes you respond 
"Fine'^ before someone asks, "How are you?") This sort of thing happens almost 
every time educators sit down to plan a conference* One asks, "Who will be the 
main speaker?" The immediate response is "Ralph Tyler". This makes his sched- 
ule a rather tight one. He was in Philadelphia last night. He was in Detroit this 
morning. This is a consequence, I think, of the question, "Whom will we have as 
the main speaker?" 

As the program indicates. Dr. Tyler is Director oft' Center for Advanced Study 
in the Behavioral Sciences; but historically, he has been more closely identified 
with the University of Chicago* He might some day return if his" Board ceases to 
be magnanimous. He took his Ph.D. in 1927, afler serving as a high school teacher 
in the science area in Pierre, South Dakota and in Nebraska. He taught at the Uni- 
versity of Nebraska, Ohio State, and North Carolina. As you kr^ow, while he was di\ 
Ohio State, he co^iducted a series of studies, which have always interested me, on 
the relationship between information and the ability to use it. At Chicago, before he 
came to the Center at Stanford, he held a series of positions. He was Dean of the 
Social Sciences Division, University Examiner, and Chairman of the Department of 
Education. He is perhaps more closely identified with the eight-year or the thirty- 
school study, of which he was director. He also has acted as Director of the- Exam- 
ination Staff of the United States Armed Forces Institxite and directed a cooperative 
study in general education, a college level study, some years ago. 

He has been consistently one of the most influential people in American education, 
and I think you will see why. He is going to talk about "Background for Recent Cur- 
riculum Developments". Dr. Tyler will speaJc frankly about some of the issues to 
which Mr. Rourke will speak later, but he will talk in such a way that you will think 
he is talking about somebody other than you. Mr. Rourke will make you imcomfort- 
ably aware of the fact that is talking about you. This is why Dr. Tyler is Director 
of the Advanced Study Center and why Mr. Rourke is not. 

BACKGROUND FOR RE.CENT CURRICULUM DEVELOPMENTS 

RALPH W. TYLER • 

The papers in the East yesterday made a good deal of the cancer of Arthur Godfrey, 
butl^m happy to see we've discovered a substitute. 



This is a regional conference, but I notice in the current issue of the Saturday Review. . 



that Wallace Stegner said there is one area in this country that is not a region* 
That is the western area of the coiuitry* It is the one region which has a world 
outlook, not even limited to the West Coast or to the United States* This is not 
true of the rest of the country. 

I had the privilege not long ago of being on a committee with Paul Cabot, the fa- 
mous leader of Boston society. Several proposals were being made, and he ob- 
jected to all of them. Finally, somebody said, "The trouble with yo?i, Mr. Cabot, 
is that you are a conservative New Englander.'' He answered, ''Yes, you know 
you Westerners always think that we are conservative New Englanders. I just 
visualize what you think a conservative New Englander is: a tall, gaunt, dour- 
faced man wearing a black, shiny* alpaca coat, sitting on a jagged rock, sipping 
vinegar, atnd contemplating rape." This regional conference will not be restrict- 
ed to any conservative point of view. It will be, I trust, representative of all 
that we are thinking of in this country. 

Any effort to review the development during the past 50 years of the srfiool cur- 
riculum of the United States encounters a confusing complexity. This is'hbt due 
alone, or even primarily, to the fact that we have no centralized control of edu- 
cation in this country, although it is true that the variations among the 48 states 
are easily marked. These local variations *Lre less pronounced than are the .dif- 
ferences of another sort. 

One may analyze the development of curriculum theory, that is, the statements 
of rationale for the curriculxim auid the related discussions which seek to explain 
it and to make it more coherent and systematic. One may examine the work of 
the persons and groups who have designed courses of study and prepared curric- 
ultun gmdes. One may assemble reports of the curriculum in operation in cer- 
tain schools to jobtain a picture of what teachers are actually teaching at a giv* • 
time in the few schools on which this kind of report is available. These three 
views will give quite different content, yet each is a significant aspect of the Amer- 
ican school curriculvun. 

Another factor in the complexity of this topic arises from the tmusually compre- 
hensive definition of the term "curriculixm** which is currently employed in Amer- 
ican educational circles. The term is not limited in this country, as it commonly 
is abroad, to refer to the oxitline of the content to be taught but is used to include 
all of the learning of students which is planned by and directed by the school to 
attciin its educational goals. This inclusive definition covers the formulation of 
educational objectives, the planning, use, and organization of learning experiences, 
and the appraisal of student learning. It also includes not only the learning activ- 
ities carried on in the classroom and laboratory, but also those at home or in 
extra-cur riculimi situations insofar as these are planned ajid directed by the school 
to attain its aims. The line drawn between the curriculum and other activities of 
the students is that which separates activities designed by the school to contribute 
to educational ends from those which are provided for recreation or for other pur- 
poses or are not part of the school^s plan. It is true that the current definition of 
the curriculum is a more adequate one for thinking about, for planning and for con- 
ducting an educational prograpi, but it does mean that a review of the curriculum 
must include a larger range\Af matters than would be required if the definition were 
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limited to the outline of content to be taught. 

To bring my task into manageable size, I have chosen to focus attention on the 
development of curriculum theory over the past 50 years with occasional com- 
ments on the ways in which courses of study and curriculum guides diverge from 
the accepted ratioiale and with still fewer occasional comments on the discrep- 
ancies between teaching practices and curriculum theory. To simplify this com- 
plex review still further, we shall examine each of three major aspects of the 
curriculiun in ttirn, treating the formulation of educational objectives, the selec- 
tion of learning experiences and the organization of learning experiences. Al- 
though the evaluation of the effectiveness of the curriculum is commonly included 
as an aspect of the curriculum itself, I shall not discuss it, since the two follow- 
ing papers deal with this sub'ect. 



The Formulation of Educational Objectives 

A major step in most theories of curriculxxm development is the formulation of 
the educational objectives of the school, that is, the goals to be attained by its 
educational program. To be sure, this is not an appropriate step in-John Dewey's 
educational philosophy in which the direction of learning is guided by careful con- 
sideration of the quality of the learning experiences. Insofar as the learning ex- 
periences provide for continuity and interaction, in Dewey's terms, the program is 
effective. His philosophy does not involve a distinction between ends and means. 
But the other chief leaders of American curriculum thought emphasize the impor- 
tance of clear objectives as the basis for planning the learning experiences and 
appraising the results^ 

Since the turn of the century, there have been several marked changes connected 
with the formulation of the objectives of the school. One of the most obvious has 
been the changed conception of the nature of educational objectives. The dominant 
educational psychology in 1900 was based on the theory of formal discipline and ex- 
pressed in terms of "faculty psychology". The mind had certain faculties such as 
memory and reason which could be trained or disciplined by proper exercise. The 
objectives of the school were stated in terms of the faculties to be trained, and the 
learning experiences were those exercises in which these faculties were engaged 
on content particularly rich in opportvmities for memorization, reasoning, and the 
like. Certain subjectSv by the very nature of their form and content; were superior 
means for cultivation of these faculties. Language, particularly Latin, for example, 
was a superior subject because the learning of it required the exercise of memory, 
while its grammatical structure provided exercise in orderly reasoning. 

With the decreasing acceptance of the theory of formal discipline and the elimina- 
tion of faculty psychology, the prevailing view became increasingly behavioral. 
Learning was then conceived as the acquisition of patterns of behavior which the 
student had not previously followed. Human behavior was defined quite generally to 
include all the reactions of an individual, his thinking, feeling, acting. 

Educational objectives are now couched in behavioral terms. An objective is a 
statement of a kind of behavior pattern which the school seeks to have the student 
develop. In the first flush of behavioral concepts, roughly from 1918 to 1925, the 
objectives were commonly stated in highly specific terms, such as ability to add 
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2 plus 3, ability to use the i^rdefinite article "an'', ability to spell "believe", abil- 
ity to recall the atomic weight of sulphur* This was a natural corollary to the pre* 
vailing associationist theory in the psy;.nology of learning. Every nxunber combination, 
for example, was viewed as a different stimxUus to which the. student was to learn an 
appropriate response. This extreme view led to the listing of nearly 3000 specific ob* 
jectives for arithmetic, and nearly 2000 for English. A student had attained the goals 
of the curriculum when he had learned to make the appropriate responses to all of the 
specific stimuli, that is, when ail of these innimierable objectives bad been reached. 



By 1925, this view of objectives had largely fallen of its own weight. On the side ot 
the teacher, it required keeping in mind far too many goals to be remembered; and on 
the side of the student, it denied the development of generalized behavior patterns 
which quite obviously were developing* The formulation of other theories of learning 
which took into accoxmt the phenomenon of generalized behavior provided terms in 
which educational objectives have commonly been stated since 1930. For example, 
in 1936 the Department of Superintendence of the National Education Association pub<- 
lished a yearbook on "The Social Studies Curriculum". Among the objectives sugges** 
ted were: (pp. 320-*340) 

1. Acquisition of important information 

2. Familiarity with technical vocabulary 

3. Familiarity with dependable sources of information 

on current social issues 

4. Immimity to malicious propaganda 

5i Facility in interpreting social science data 

6. Facility in applying significant facts and principles 

to social problems of daily life 

7. Skill in investigating social science problems 

8. Interest in reading about social problems and in 

discussing them 
. 9* Sensitivity to current social problems 

10. Interest in human welfare 

11. The habit of working cooperatively with others 

12. The habit of collecting and considering appropriate 

evidence before making important social decisions 

13. Attitudes favorable to social improvement 



These obviously present a conception of generalized behavior. However, although 
they avoid the piecemeal aims of highly specific objectives, chey may be as limited in 
their value for guiding teaching as the earlier statements of objectives in terms of 
faculties to be developed imless each of these thirteen objectives is clearly enough de* 
fined to have meaning for the teacher so that he can easily think of concrete illustra- 
tions of the general aims. The developments since 1935 in the conception of the na«- 
ture of educational objectives have largely focused on defining in concrete terms aims 
which are expressed at a similar level of generality as those above. These efforts 
have been applied to definfxvg^the kind of behavior implied by such general terms as 
"understanding", '^applying principles to concrete problems", "ability to interpret 
reading material", and to indicating the range of content to which each kind of beha** 
vior is to be applied. Thus, the objective "to develop imderstanding of the basic 
concepts of physiology" has been defined from the standpoint of behavior and of con- 
tent. The behavior "imder standing" is defined as "the ability to recall the concepts. 
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to state them in one^s own words» to give illustrations of them> to recognize iHus* 
trations given by others^ and to compare and contrast related concepts." The con* 
tent termed ''the basic concepts of physiology" is defined by listing some two score 
concepts which these curriculum makers have selected as basic to this science. This 
kind of definition helps greatly to clarify the aims of the curriculum so that they can 
actxially be utilized in planning and conducting an educational program in terms of the 
prevailing conception of the ^psychology of learning. 

A second marked change in the objectives of the American school curriculum has 
been in the sources used to derive the aims« To some extent all of the five major 
sources have been used in every period of American history^ but at a given time cer- 
tain sources are dominant in their influence, while others are given only minor at- 
tention. Between I9OO and 1918, the judgments of subject specialists and the prevail** 
ing conception of the psychology of learning were dominant in formulating objectives. 
At the high school level, the Committee of Ten used sub*-committees of mathemati^ 
cians, historians, language scholr rs, and the like to outline the objectives of Si'tcond* 
ary school instruction in these fields. Although the prevailing educational philosophy 
had already emphasized knowledge and skill for the layman as a major aim of the 
American high school, this was given little attention in deciding on objectives. No 
studies were made of the needs of society nor of the needs of students to help in ider • 
tifying appropriate objectives. 

As a result of the success of job analysis in building vocational curricula during 
World War I, the process of formulating objectives from 1918 to 1933 leaned heavily 
up n job analyses, activity analyses, word counts and other techniques for identifying 
the demands made on the individual by contemporary social life. At this time, cur* 
riculum makers also gave attention to the notions of educational psychologists as to 
what behaviors could be taught. However, during this period little attention was given 
to the prevailing social and educational philosophy regarding the characteristics of the 
good man and the good society. The opinions of subject specialists were given much 
less weight than in the previous period. 

From 1933 to 1945, studies of children and youth served as a majoi- source of sugges*-* 
tions for objectives. With an emphasis upon the responsibility of the school for meet-- 
ing the needs of children and youth, curriculum commissions drew upon child study 
data and reports of adolescent studies to derive objectives. This largely coincided 
with the prevailing emphasis in educational philosophy, and to some extent the work of 
educational psychologists was used* But the use of studies of social demands was no<- 
tab'ly less than in the decade previous, while the opinions of subject specialists played 
a very minor role. 

Since the second World War, the £hift in emphasis among the '?ve kinds of sources has 
been marked. Primary attention is currently given to the opinions of subject specialists, 
particularly in mathematics and science. Very little weight is currently given to studies 
of the learner, but the specialists are asked to outline what they believe to be important 
potential contributions of their fields which will be of value to laymen as well as persons 
planning to specialize in the field. In this respect, the emphasis is different from that 
in 1900. Today some attention is also being given to an examination of social demands, 
and to a lesser extent, to the current conception of the psychology of learning. ^^'l^ch 
le&s \Xise is made today of studies of the learner than was true 15 years ago« In general^ 



the shifts which have taken place in the primary sources used to derive educational 
objectives most closely parallel the changes which can easily be seen in the state-- 
ments of objectives appearing in courses of study and in curriculum guides. Because 
the actual practice of teaching depends so largely on the habits and outlooks of the 
thousands of American school teachers, the shifts in practice are not so easily dis- 
cerned, 

A third marked change has been in the range of objectives which the American schools 
have not only accepted for themselves but have actively champic ned. At the turn of 
the century, there was a sharp difference between the claims made regarding the 
schools' general contribution in promoting citizenship and character and the working 
objectives cf the curriculum which were focused on knowledge and skills and intellec- 
tual disciplines. The development of many basic attitudes, values, interests and'Kab- 
its was recognized as a primary function of the home and church, and for those habits, 
attitudes and skills relevant to work, the employer was expected to play a strong role. 
The school today commonly lists the whole range of educational goals required for the 
induction of young people into effective adulthood. It includes object wes relating to 
home life, personal-social relations, civic life, occupations, ami the like. It includes 
not only knowledge and intellectual abilities but interests, attiiud^*^. social and re- 
creational skills. Frequently, too, there is no indication of relative weighting. De-- 
veloping social skills and a cooperative attitude appear to be viewed as jobs as impor- 
tant for the school as developing understanding of basic concepts of science and the 
social studies or the acquisition of the skills involved in reading. 

Since the level of learning required of people today is a high one, a major problem in 
education is to select wisely among all the possible goals the important tasks which 
the school can do well and to concentrate its energies effectively. Since the total edu- 
cational job is very great, the home, the church, the employer, and the other potential 
educative agencies of the community need to be encouraged and strengthened to take 
their shar^ while the school concentrates on the things it can do best, and in many cases 
the things that only the school can do. Hence, the present shift in school objectives is 
toward a more discriminating selection, toward the kinds of learning which involve in-- 
tellectual skills, which require sequential experiences to reach the necessary level of 
competence, and which involve concepts and principles that are not apparent on the 
surface and for this reason are not likely to be learned through the guidance of laymen. 
This shift is likely to reduce the great range of objectives, and to diminish the emphasis 
upon social adjustment and similar goals which fail to recognize the importance of indi- 
viduality and individual creativity in responding to experience and solving problems. 
The increasing emphasis upon understanding and thinking as kinds of objectives, wath 
lessened stress upon attitudes and habits as primary goals, may help to revive the con- 
ception of the individual who controls his feelings and actions in terms of his knowledge 
and thought rather than one who simply seeks to express ''acceptable** attitudes and 
feelings and to do the "proper'' thing. This is a shift in objectives which will be inter- 
esting to observe. 

The Selection of Learning Experiences 

Among the changes taking place in the learning experiences provided by the American 
schools, those in the prevailing notions of the nature of learning experiences are par- 
ticularly significant. At the beginning of this century, the term was not used. Exercises, 
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assignments, examples, problems were the words commonly employed to designate 
the learning tasks set for students outside of the class session, while the term "re- 
citation" was used to refer to the oral responses expected of the student in the class. 
No mention was made of the student's mental reactions in the class, although it was 
clear that he was expected to pay attention, that is, to watch and listen to the teacher's 
presentations. When I -began to teach more than 35 years ago, we had to file lesson 
plans for each week in advance. These plans outlined the content to be covered, what 
the teacher expected to do, and the out-of-class assignments to be made. The focus 
of planning was on the teacher. 

John Dewey and other educational leaders gave wide publicity to the increasing psy- 
chological evidence that learning could be most readily interpreted in terms of what 
the learner was doing. It was his reactions that he learned - not the teacher's. The 
teacher's role was to stimulate, guide and reward the learner as he carried on the be- 
havior which the school sought to teach him. This view placed attention upon the activ- 
ity of the learner as the basic factor in attaining educational goals. By 1925, both 
writings of theorists and curriculum guides were commonly using the term "learning 
activities" to refer to the basic elements of the teaching-learning situation. Courses 
of study were listing reading activities, listening activities, study activities and lab- 
oratory activities in outlining the day-by-day program of the school. 

By 1935, curriculiun writers were pointing out certain limitations in the concept of 
learning activity. For example, two students might both be reading an historic account 
of the California "Gold Rush", yet each might be carrying on quite different mental re- 
actions and making different emotional responses. One might be thinking of the excite- 
ment aiid challenge involved in the long wagon haul the pioneers made in crossing the 
country, thrilling himself as he imagined the Indian encounters. The other might be 
thinking of the rough, lawless life of the early mining community, wondering why people 
would leave the comforts of civilization to live in such trying conditions. In terms of 
the course of study, both were engaged in the same learning activity, but each was hav- 
ing a different experience and to that extent was learning something different. This kind 
of analysis led to the adoption of the term "learning experience" to refer to the reactions 
of the student in the situation. In 1936, Dewey's book on Education and Experience clar- 
ified this concept further by emphasizing the notion that "experience" involves the inter- 
action of the individual with the situation. This interaction involves some mutual effects, 
the individual modifying his reactions in terms of the demands of the situation, and also 
modifying- the situation through his reaction to it. Today almost all curriculum writers 
use the term "learning experience" and they seek to plan the learning situation so as to 
give direction to the experience the student has, that is, to his internal perception of the 
situation and his own interaction with it. This requires consideration of what the learner 
brings to the situation, what it will mean to him, and how he is likely to respond to it 
mentally, emotionally, and in action. 

Beginning with James and Thorndike and exercising increasing inHuence in recent years 
is the conception of the learning situation as one which should provide for certain essen- 
tial conditions of learning. Thorndike's earlier work emphasized two conditions - exer- 
cise and effect. Current curriculum guides mention such conditions as motivation, op- 
portunity for practice, guidance of desired behavior, provision of satisfaction when de- 
sired behavior is elicited, and the like. Hence, some of the current courses of study are 
pointing out the need to consider these conditions in selecting the learning experiences for 
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a particular class group from a larger list of suggested ones. 

A second marked change in learning experiences can be found in their range. Al- 
though the Sloyd movement had influenced forward-looking American schools in the 
late 1800s to introduce .nanual training, not as vocational training but as a means 
of "learning through the hands*', most of the learning exercises employed at the turn 
of the century were verbal ones^ ^Listening and reciting, reading and writing repre- 
sented the ways of learning the "academic subjects", except for the laboratory periods 
in high school science. Even the laboratory exercises were heavily verbal, with de- 
tailed instructions in the manual and a formal plan for writing up each "experiment". 
1 p work in geography and field work in biology were strongly recommended by the 
writers of the period from 1905 - 1915. Most courses of study advised having chil- 
dren make maps and locate points of geographic interest on them. At this time, too, 
high school botany courses typically required the student to collect and identify 50 or 
more plants. 

By 1910, high school agriculture was widely offered in rural areas. These were the 
first courses to introduce the project, or "student-initiated" enterprise, which, it was 
hoped, would help him to understand and to apply the knowledge he was gaining in the 
course. The use of projects spread to other fields and to the elementary school, thus 
providing a much wider range of learning experiences than schools had commonly used. 
The writers who urged the introduction of projects conceived of them as involving a 
range of experiences as broad as life itself, but in the actual use of projects in the 
schools, activities involving the construction of objects have been predominant. Many 
teachers think of a project as making, growing, or producing some physical object. 
The extended inquiry which Dewey thought had largest potentialities as an educational 
project is rarely found. The intellectual learning experiences are frequently quite 
minor to the physical manipulations required to complete a "construction project". 

During the depression, with its great reduction in opportunities for remunerative work 
for youth, many secondary school leaders recommended the addition of work experience 
to the high school program. Although only a small minority of high schools introduced 
work experience as part of the curriculum, some developed well-planned programs 
which involved using a wide variety of work activities as means for attaining educational 
objectives related to science, social studies, mathematics and English, as well as vo- 
cational fields. 



The greatest impetus to extending the range of learning experiences has been the tech- 
nological development in communication. Lantern slides were m use at the turn of the 
century but were not found in many schools ♦ At best they served only to extend the num- 
ber of pictures which could be employed, to add concreteness, or to give variety to the 
teacher^s presentation. The perfection of the motion picture, however, made it possible 
to analyze movements, to show time and space relationships much more graphically, and 
to increase the sense of reality in dealing with many subjects which require vicarious 
treatment. The addition of the sound track heightened the sense of reality and added an • 
other dimension of analysis. The sound-slide film gave some of the features of the sound 
motion picture in a more economical form, but it lacked the distinctive assets of motion. 
The television set made possible instantaneous viewing of events in a fashion much like 
the motion picture but with a further sense of the reality of the event, due to the viewer 
realizing that it is taking place at the same time he is seeing it. These technological 
O ^ developments have gone far in removing the physical limitations to providing as wide a 
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range of learning experiences in the school a^ those of life outside- But much of 
the comprehensive, effective development of these potentialities lies ahead. They 
still represent a small per cent of the learning experiences provided by American 
schools* 

The selection of learning experiences so as to provide for individual differences 
among students is another respect in which changes have taken place in the last 50 
years* Attention to individual differences has been accentuated by two factors: the 
psychological studies which have identified the extent of differences among schoolsi 
among classes and among students in the same class; and the increased visibility of 
individual differences brought about by the enrollment in the school of children from 
heterogeneous ethnic groups and social classes. There are few teachers now who 
fail to recognize a variety of differences among the students in their classes - dif- 
ferences which affect interests, meanings, efforts and outcomes in school work. 

Typical devices to provide for differences among students have involved adaptations 
in the time given for completing learning exercises, or variations in the exercises 
themselves, or both. The first type of adaptation requires a plan for students to 
work at varying rates. Among the early developments were the San Francisco, Dal- 
ton and Wimietka plans, all of which involved orgajiizing the school day into two parts, 
one for group activity and the other for individual work. These plans also reqmred 
the development of a series of assignments with full directions on paper so that the 
students could work as individtials on different assignments at the same time. As a 
student took an assignment, it became his "contract", which he undertook to finish 
before he went on with another assignment in the same field. He might, therefore, 
complete his assignment much earlier or much later than the average. 

Adaptations of the leaning experiences themselves were first found in courses of 
study which marked some of the exercises as those to be required 3f all students and 
others as optional for the better students. By 1915, this was common among Amer- 
ican schools. By 1925, a number of cities had introduced "ability grouping" in which 
the course of study was differentiated in such fields as reading and arithmetic into 
three levels - the superior, the average, and the slow sections. These three courses 
of study differed in the time provided for learning exercises and to a lesser extent in 
the nature of the exercises. In reading, the amoxmt of material dealing with personal 
and social activities of children was greater in the slow sections, while the adult ma- 
terial was greater in the superior sections. In arithmetic, more concrete objects 
were counted and compared in the slow sections than in the others. 

The use of individual projects was also a means of adapting to the individual student's 
interest and ability. This was recommended in courses of study as early as 1915. 
Learning exercises carried out by small groups (two to ten students) were first em- 
ployed in the late 1800s to compensate for inadequate laboratory equipment. The 
apparatus was insufficient to provide opportxmity for every student working individually 
to carry out the assignment. By 1930, small group projects were being used by many 
schools as a manageable means of providing for individual differences. The projects 
themselves could differ in the rigor of their intellectual demands, and the division of 
labor among the students in the small group could adapt further to the abilities and in- 
terests of the individual. Unfortunately, all too often the slowest learner was given 
some handwork which involved little if any new learning. By 1950, with the publica- 
Q tion of research on the psychology of small groups, educational writers were recom- 
mending the use of small group projects as a means of heightening motivation and 
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increasing the amount of meaningful learning activity. Since 1948, the attention 
of educational leaders has focused increasingly on the education of the gifted stu- 
dent. This has led to emphasizing learning experiences which require greater 
understanding, skill, or effort than those usually provided in the course of study. 
It has also stimulated some schools to develop learning experiences that can be 
carried on as independent work. 

The most typical development in the past 20 years found in ::ourses of study to 
provide for individual differences has been the listing of a large number of sug- 
gested learning experiences from which a given teacher may select ones particu- 
larly appropriate for his class as a whole or for groups or individuals within the 
class. ^ The uniform lesson plan so common when I started to teach is almost un- 
known now. Most curriculum guides include a discussion of hbw to select from 
among the large number of learning experiences suggested in the course of study 
those which are likely to be most effective for students with varying backgrounds 
and abilities. 



Important educational objectives involve patterns of behavior of such complexity 
that they can be developed only gradually over considerable periods of time. For 
example, the ability to read critically and to make comprehensive interpretations 
of what one reads is not acquired in a few brief lessons- To understand the basic 
principles of science and to use these principles in explaining the biological and 
physical phenomena round about us requires a variety of related experiences ex- 
tending over many hours. If the development of such complex behavior patterns 
as these is left to isolated or unrelated periods of learning, adequate achievement 
is impossible. Hence, a major phase in building a curriculum is to work out an 
organization of the many, many learning experiences required so that the student 
develops these complex behavior patterns gradually, day by day, and relates them 
to others so as to have an increasingly unified understanding and a well integrated 
command of essential skills « 

The purpose of organizing learning experiences is to maximize the cumulative 
effect of the large number of learning experiences required to develop complex be- 
havior patterns. Three criteria are commonly considered as standards to be met 
by a well organized curriculum, namely, continuity, sequence, and integration* 
Continuity refers to the reiteration of the desired behavior through the many learn- 
ing experiences used. Sequence refers to the gradation of the learning so that each 
subsequent experience not only builds on previous ones but goes beyond in order to 
require a higher level of skill or a broader or deeper aegree of understanding. In- 
tegration refers to the relation of what the student is learning in one field to what he 
is learning at about the same time in other fields. A broader and deeper under- 
standing is facilitated by comprehending the relation among the various concepts, 
facts and principles being studied, and a more adequate command oi basic skills is 
achieved as the relation of these skills to one another is^ seen. 



One surprising fact about curriculum development in ^he last 50 years has been the 
limited attention given to the theory of curriculum organization. Other than the com- 
rnon-sense notions of these criteria and of such rule-of-tbumb principles as "learning 
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experiences should proceed from that which is known to that which is unknown, from 
the simple to the complex, from the easy to the difficult'S no new formulations have 
been made since the time of Herbart and James. This is an area crying for substan- 
tial theory to be tested in practice and to provide ct guide for practice. 

At the more specific level, developments in reading and in foreign languages have 
been most marked. In reading, continuity and sequence are commonly achieved through 
carefully controlled vocabulary development, adding new words gradually and^system- 
atically, and through the control of sentence structure in the reading materials, begin- 
ning with simple declarative sentences and moving gradually to compound and complex 
ones. Integration is sought both by relating the reading material to the common activ- 
ities of the children and by introducing work-type reading in the other subjects on a 
gradual basis • A similar scheme of organization is commonly followed in the foreign 
languages. 

In arithmetic, the development of skills is usually facilitated through an organization 
which begins with learning experiences involving addition and subtraction, then multi- 
plication and division, then common fractions and .decimal fractions. No explicit scheme 
of organization for concept development in arithmetic can be found in the current courses 
of study. The content of arithmetic problems has changed greatly since 1900. Beginning 
about 1920, studies were made of the kinds of problems commonly encountered by chil- 
dren and adults. Typically, aritlimetic courses now order the problem content in terms 
of frequency of crcurrence of the problems outside of school and in terms of the age level 
at which this kind of problem is commonly encoxxntered by children. 

The typical high school curriculum in mathematics has changed little in the past 50 years 
so far as organization is concerned. Tenth grade geometry builds little, if at all, upon 
algebra. Advanced algebra and solid geometry in the eleventh grade have little sequen- 
tial relationship to tenth grade geometry, and trigonometry in the twelfth grade does not 
provide a clear sequence for the eleventh grade work. The so-called ^'modern mathemat- 
ics program" which is now getting under way with the sponsorship of the mathematical 
organizations should provide a much better organized curriculum for high school mathe- 
matics. 

In organizing the so-called content fields, like the sciences and the social studies, ma- 
jor attention has been given to the ordering of content rather than behavior. At the be- 
ginning of this century, science was not commonly taught in the elementary school, while 
in the high school, botany was most frequently offered in the tenth grade, physics in the 
eleventh, and chemistry m the twelfth. By 1920, general science was offered as the in- 
troductory science course in more than aia- fourth of the high schools, and now it is 
taught in almost all schools in the eighth or ninth grade, with biology in the tenth and 
physics and chemistry, where offered, being placed in the eleventh or twelfth grades. 
The content of general science is usually selected to relate to the scientific phenomena 
most commonly observed by children. The content of biology is usually chosen to explain 
the human body, the maintenance of health and the conservation of natural resources. The 
orga.nizing notion here is to begin with phenomena which are common in the student's en- 
vironment and in which he is likely to be interested. The advanced science courses, phy- 
sics and chemistry, deal with the more abstract principles, which are thought to be less 
common and more difficult. The organization of these two courses has not greatly changed 
in the past 50 years. These illustrations in the field of science indicate the attention given 
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to organizing" tire content dealt with in the learning experiences, but no similar 
effort has been made to organize the behavior, that is, the skills and abilities to be 
developed* 

This is also true for the social studies. The changes taking place in their organ- 
ization have been charges in the ordering of contents The most common sequence 
of content in the social studies is to begin with the community, then the state, then 
the nation, and finally the world. There is little evidence to indicate that this is 
sequential in terms of difficulty in learning. 

Thus far, we have been reviewing the continuity and sequence of learning experiences 
in the contenMields. The problem of integration, that is, how to relate learning ex- 
periences so as to aid the student in seeing the relations between what he is learning 
in One field with what he is learning in another, has been attacked most commonly 
through changes in the structure of the curriculum. In 1900, the elementary school 
curriculum was composed of 10 or more specific subjects like reading, writing, spell- 
ing, arithmetic, geography, history, nature study, hygiene, music, drawing. Now, 
the typical course of study includes reading and the language arts, arithmetic, science, 
fine arts, health. This reduction in the number of subjects has been accomplished by 
building a more closely related series of learning experiences in language, in which 
reading, writing and spelling are involved; in social studies, where geography and 
history are interrelated; and in the fine arts, where music, drawing, and painting are 
brought together. 

In the high school, the broad fields of English, mathematics, science, social studies, 
foreign language and fine arts. have frequently replaced more specific subjects, and in 
some cases the core curriculum has been developed, which provides a large structure 
for learning experiences that occupy from one -third to one -half of the high school stu- 
dent's day. Since these larger structures are usually planned as courses rather than 
several separate sub-courses, there is opportunity for better integration. Typically, 
however, the only principle of integration which has been explored is to bring together 
the content and skills needed to deal with each of the student "problems" which provide 
basic units of the course. This principle does not always provide for the necessary 
continuity and sequence nor for all of the more helpful relationships among the fields 
which are involved . In many cases, a particular problem involves knowledge or skills 
from certain fields in only a minor degree and does not suggest the more significant 
ways in which these fields are related. 

It is clear, after reading the works of curriculum theorists and examining courses of 
study, that the past 50 years have not been a time of great development in the organ- 
ization of learning experiences. In this respect, curriculum changes have been rel- 
atively few. The careful, systematic work done in the field of reading is a shining 
exception. The arousal of interest and stimulation of thought among secondary school 
teachers who have worked on the ^ instruction of core curricula suggest the great in- 
tellectual resources available under effective leadership to attack fundamentally and 
systematically the problem of developing a better organized curriculum. 

Summary 

This review of changes in the curriculum of the American schools during the past 50 
years has touched several high spots, but it has not presented possible explanations 
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for the kinds of changes noted. It is probable that many of these developments can 
be understood in terms of the tasks which the American schools were facing at these 
different periods and the ideas prevailing in the field of psychology which school 
leaders found when they sought assistance from scholars in attacking critical school 
problems* 

In the period prior to World War I, the elementary school was steadily growing to 
include a larger per cent of the children of age 6 to 14. The critical task was to 
teach the three R*s to children of immigrant parents and those from the working 
classes. This required a re -examination of the psychology of reading anl". arithme- 
tic, for the usual background of interest in and experience with language and childrerfs 
stories could not be taken for granted, nor could early experiences with number con- 
cepts in the home be assumed for all children* It is not surprising that in this period 
long lists of specific objectives for these subjects were worked out and that special 
attention was given to children's interests and abilities in devising learning experi- 
ences. 

World War I ushered in a 15-year period when the economy developed rapidly and 
workers were in demand. No wonder that job analyses and other forms of analyses 
of social life were used to identify more definitely the demands which the individual 
would be expected to meet, and to use this source in setting objectives* From 1930 
until World War II, the great depression increased rather than decreased the number 
of youth in school because of the limited opportunities for employment* But the same 
limitations in jobs made the analysis of social demands a less relevant source for 
educational objectives than a study of youth themselves to find needs and potentialities 
that might justify educational effort when no great social demand was apparent. This 
period also was a difficult time for many secondary schools because youth saw no fu- 
ture and had no interest in deferred educational values. In their eyes,* school work 
had to be justified immediately in terms of its interest and meaning to them* Hence, 
the devising of learning experiences of immediate interest and relevance to students 
and their organization around student problems helped to meet this pressing problem. 

Since World War II, the insatiable demand for technical! / trained people hi\s focused 
attention on the opinions of subject experts as a source of objectives and has given 
greater emphasis to knowledge and skills. The current demand for highly competent 
professionals and technicians has increased the interest of the schools in the educa- 
tion of gifted children and in devising a better organized curriculum to reach high 
levels of achievement. The shortage of teachers has furthered the demand for tech- 
nological devices that increase the range of learning experi'=*nces, such as television 
and the sound motion picture. 

Throughout the last 50 years, the schools have been pressed by continuing conditions 
which create critical problems that cannot be solved without further curriculum de- 
velopments. The first of these is the rapid change in technological development and 
social life, which requires a continually increasing level of education on the part of 
our people. The second is the increasing proportion of children and youth who are 
sent to the schools for education. The third ie the dislocation in other educational 
institutions, the home, the church, and the neighborhood, which rapid social change 
has engendered. The educational needs of today and the immediate future are greater 
than ever before • American education has done an amazing job in getting almost all 
children and youth in school and providing schools for this immense number. The 
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schools have been astoundingly successful in building confidence on the part of the 
public in the capabilities of education in building our civilization. The time has 
come, however, to recognize realistically the magnitude of the job, to identify the 
objectives which the schools can best attain, to encourage the home, the church 
and other institutions to undertake the tasks appropriate to them, to devise learn- 
ing experiences clearly relevant to the schools' proper objectives, and to work 
out an organization of the curriculum which aids the students in attaining a high 
level of educational competence. These steps still lie ahead of us. 

CHAIRMAN COLADARCI: Does anyone have a question or a response for Dr. Ty- 
ler? As I told you earlier, he voices his criticism in such a way that it sounds 
like he is talking about somebody else, so it's safe to speak. If there are no ques- 
tions at this time, you may bring up questions later during this session. 

Our next speaker, Mr. Rourke, is a threat for me. I spent some time with him 
last night at dinner. Usually, I have a lot of fun with a person who has such a de- 
lightful sense of humor because I enjoy the interchange. But last night I seldom 
had a chance to say anything, and he always had the last word. I warned him that 
I would get even when I introduced him. He reminded me that he would be speaking 
right after I did, which has silenced me again* 

Let me note for you that Mr. Rourke is from Kent School, which is a small insti- 
tution in Connecticut. I don't believe the recent canard that some organization on 
the £ast Coast has sponsored a contest, the first prize for which is a one-year 
scholarship to Kexit School. This sounds tremendous, except when one notes that 
the second prize is a two- year scholarship to Kent School. 

Mr. Rourke is a Canadian. He graduated from Queen*s University in Kingston, 
Ontario, in 1927 with honors and a gold medal in mathematics, the field which he 
represents. He had graduate training at Harvard in '29 and '31, where he took a 
Master's in mathematics. He was at Pickering College in Newmark, Ontario, 
from 1928 through 1947, where he ended up as Head Master and, between-times, 
a math teacher and Assistant Head Master. He has been at Kent School since 1953 
and is head of the Department of Mathematics and Science. Mr. Rourke has taught 
from time to time at other places: Queen's University, University of Alberta, and 
Teachers Ccllege of Columbia, where he is now completing what is euphemistically 
called a sabbatical, and during which time he has worked very hard with the Com- 
mission on Mathematics. He is going to speak about this work. 

The man has had a distinguished career in his field, and his career has been a very 
influential one. He is one of the persons involved in Dr. Tyler *s reference to cur- 
rent developments in the structuring of mathematics education. He has been an 
executive of the Canadian Mathematical Congress, a member of the Advisory Com- 
mittee, School Mathematics Study Group, Director of the National Council of Teach 
ers of Mathematics, and Executive Director of the Commission on Mathematics for 
the College Entrance Examination Board. 

He is co-author of several texts. I won^t note them all to you, but I want to read a 
few* titles, because there is a very interesting and non-accidental structuring of the 
order in which the texts are listed. The first one is An Advanced Course in Algebr 
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the second, F;^ne Trigonometry and Statics ; and third one is Mathematics for Can -^ 
adians, which isTn six volumes. The last one, which follows immediately on the 
six volximes, is called Vfhy Study Mathematics ? - a question which I presume he 
will aiiswer. 

Mr. Rourke would like to present to you the kinds of thinking that have gone on in 
the Commission on Mathematics, a report of which has recently been issued, I 
shudder to think what else he might say. I have only the advantage of knowing that 
I will return. Mr, Rourke! 

RECENT CURRICULUM DEVELOPMENTS IN MATHEMATICS 

ROBERT E. K. ROURKE 

I was informed that he would get me off with one foot on a banana peeling, and you 
can see what has happened. I'm glad he spared me the indignity of referring to Kent 
School as the school with the micronite filter. As a matter of fact, I didn't think the 
introduction was too bad. If I thought he could have delivered lines like that, I would 
have written more of it for him. I just want to say that I was told that Dr. Coladarci 
was the poor man^s Groucho Marx and that I should stand back and get ready. But I 
want to say. Sir, that I found you very kind and generous, and not nearly as offensive 
as your friends told me to expect. 

This morning I should like to give you some notion of what has been going on in the 
field of high school mathematics. In particular, I want to discuss the work of the 
Commission on Mathematics of the College Entrance Examination Board. 

Anyone who can read a newspaper is aware of the fact that during the last few years 
there has been a ferment in the field of mathematics, and that this ferment has ex- 
tended all the way from the first grade to the graduate school. There is sweeping up 
to be done on all the floors of the mathematical skyscraper. Not long ago a speaker 
addressing a meeting of teachers and mathematics educators said: "It's a great joy 
to speak to a group of people who are working in a field in which nothing has happened 
in 500 years. " He couldn't be further from the truth if he were trying for an inter- 
collegiate prize. I think that in the whole field of knowledge, with the possible excep- 
tion of some branches of physics, there has been no area that has undergone such 
lively and even spectacular development as mathematics has undergone in the last 50 
years. A ntimber of projects have been concerned with the implications of this de- 
velopment for high schools, but the one with which I am most familiar is the Commis- 
sion on Mathematics. Consequently, most of my remarks will have to do with the work 
of this group. 

The Commission on Mathematics was called together by the College Entrance Exam- 
ination Board in 1955* Kindly note the date. There were no sputniks in the air at 
that time, and we are not a product of the post-sputnik panic. Too often in this country 
we panic under strain. During my recent visit to Russia, it became clear to me that 
Russians expect the American people to panic. I wonder how many of Mr. Khrushchev's 
decisions are postulated on this assumption. 

There is a relevant story about a Chinese engineer who was talking to an American 
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engineer. The Chinese engineer said, *'You know, we admire you American en- 
gineers. You are the greatest engineers in the world. What magnificent bridges 
you build f What magnificent highways! But there *s one thing that baffles us. We 
don't know why you panic so easily." ''Well," the American replied, "I don*t see 
the justification for that remark." To which the Chinese added, "Let's just think 
of a typical situation in my country. Now, in China, for example, when we want 
to put a tunnel through a mountain, it's an exceedingly simple operation. We put 
5000 people at one side of the mountain and we start them digging; then we put 
5000 people at the other side of the mountain and start them digging. If they meet 
in the middle, then we've got a tunnel." "Ah," said the American, "but suppose 
they don't meet in the middle." "Then," replied the Chinese, "we've got two 
tunnels. " 

Note, then, that in August 1955, Mr. Frank Bowles, President of the College 
Boai^d, called the Commission together. He wanted us to take a fresh look at the 
program in high school mathematics in view of the new and increasing demands 
the Twentieth Century was making upon this subject. The Commission, composed 
of school and college teachers and mathematicians, met in Ann Arbor, where we 
soon came to the conclusion that the time for a change was long overdue. And I 
should like now to tell you why we came to this conclusion, what we decided to do 
about it, and how we went at the job. 

Why should there be a change in the program of school mathematics at this par- 
ticular time? Is there a real need for a change, or is this just another periodic 
eruption like those of the Thirties, when the teachers of mathematics were reg- 
ularly going from crossroad to crisis? In those days, -unfortunately, we teachers 
were rather ineffective salesmen for our subject. Or perhaps the American peo- 
ple were not ready to face the facts concerning the role of mathematics in our 
society. Conditions are very different today. The urgency is there for all to see. 

First, the development of new mathematics has been fantastic. A mathematician 
said not long ago that since the year 1900, the amount of new mathematics that 
has been discovered exceeds by the factor 10 the mathematics known previous to 
that time. It takes some 1200 pages a year just to list the results of new mathe- 
matical research. 

Some will say, "Yes, but this development is at the frontier. What in the world 
has it got to do with the high school program, with the dementary school program, 
or with freshman-sophomore programs at college?" The answer is that a great 
deal of it has nothing to do with these programs, but the answer is also that parts 
of it have very serious implications for those who teach in school and college. 

I am a teacher, and I have been a teacher all my life. And I speak to you as a 
teacher when I say that this great wealth of new mathematics and new applications 
of mathematics can help us to offer our talented students programs, that are ex- 
citing, significant, and challenging. This fact was recently underlined by some 
documents sent to me by the Organization for European Economic Cooperation. 
Now, this is not a mathematical organization. It is not even an educational organ- 
ization. Its job is to assist in the economic recovery of the western world. And 
the documents report that O.E.E.C. has encountered a roadblock in carrying out 
its job: it cannot do effective work in terms of economic recovery in the western 
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world without significant changes in the teaching of high school mathematics. 

Some people have concliTded that all the significant changes and all the most spec- 
tacular uses are found only in the newer mathematics. This is simply not true. 
Some of the most startling achievements have been made with the aid of mathemat** 
ics long since known, a branch of mathematics called analysis. The magnificent 
work done in connection with the International Geophysical Year in the ionosphere, 
in dealing with deep ocean currents, and in studying the earth's magnetic field 
all these results stem from applications of analysis. The old stuff is still going 
great guns. 

But mathematics has a lot of new customers. When I graduated from college, I 
could do one of two things. I could be a teacher of mathematics, or I could be an 
actuary. There were no other jobs for me. Today, when I try to get people for my 
department at Kent School, I find myself in competition with Westinghouse and Gen- 
eral Electric. New uses of mathematics have opened up a great *many new jobs. 
Although the physicists and the natural scientists are still our best customers, we 
have important new customers who are demanding attention. For example, the so- 
cial sciences are using mathematics in increasing anioxints and in great diversity. 
A professor of social science told me not long ago th^.t, in many social science 
courses, you can^t get very far without the ability to handle a good course in prob-* 
abilistic thinking. Probability and statistics, the mathematics of uncertainty, are 
good, solid mathematics. 

And we are not travelling on a one-way street: the social sciences contribute prob- 
lems that le'ad to the development of new mathematics. For example, the theory of 
games» one of the most promising of the new branches of mathematics, was sug- 
gested by problems in economics related to competition and cooperation. I could go 
on to cite examples from business and industry. You are familiar with words such 
as '^operations research", "automation", and "quality control"* These indicate some 
thing of the scope of the new applications of mathematics. 

We must face frankly the fact that we have come intc a new technological era. Let 
me quote from the Rockefeller report: "First, the crisis in our science education is 
not an invention of the newspapers, or scientists, or the Pentagon. It is a real cri- 
sis. Second, the U.S.S.R. is not the 'cause* of the crisis. The cause of the crisis 
is our breath-taking movement into a new technological era. . . The immediate impli- 
cations for education may be briefly stated. We need an ample supply of high caliber 
scientists, mathematicians, and engineers. * * We need quality, and we need it in 
considerable quantity. " 

I do not wish to raise the bogey of Russia, but I must state at this time a personal 
conviction. I believe we are engaged in a mortal combat. If you had followed me 
around the schools of Moscow, Leningrand, and Kiev last fall, I think you would have 
shared this conviction. Never have I seen young people and teachers with such well- 
focussed motivation, with such determined drive. The secret ingredient of the Rus- 
sian program is work. I saw a nation with its sleeves rolled up. Some of their teach 
ing methods are almost pre-Cambrian. But the results are surprising. I was im- 
pressed particularly with their very gifted students. At times, I was almost forced 
to the conclusion that the really gifted student is impervious to all pedagogical abuse. 
He just needs a library. After going from class to class and seeing the intensity with 
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which these young people and their teachers are workingt I simply cannot come 
back and speak comfortably to Jerusalem. 

If we are going to meet the challenge of this new age, in some way or another 
we've got to provide more challenging programs for our talented - not only the 
talented in mathematics and science, but for all our talented. I don't want to 
give the impression that I would li'ke a world filled with mathematicians. Heav«- 
en forbid! I can't imagine anything more disagreeable. Fortunately, the Crea** 
tor has made this impossible. But for the college •capable we must get pro«- 
grams that are challenging and exciting. Believe me, we have exciting mathc-> 
matics today. In my high school, I haven't for some years been able to get any** 
one in the Student Guild to give a paper on traditional mathematics. I find my- 
self saying, ''This traditional mathematics has some fascinating ideas. Don't 
overlook them. " But there is no question that the ideas that excite gifted yoxmg 
people the most are those of the new mathematics. 

These then were some of the reasons why we on the Commission felt that some** 
thing must be done r«» and that, with all reasonable speed, for the developments 
of mathematics, iw growth and its uses, have far outrun the curriculum. But 
what to do? Well, we decided to select from the large problem a part that we 
felt qualified to tackle, hoping to find a good first approximation to a solution. 
So we aimed to improve the mathematics program of grades 9 through 12 for the 
top 15 or 20 per cent, which we call the college-capable. Now, this does not im^ 
ply that we think the non-*college«-capable are not important, or that grades 1 
through 8 are not important. It does not imply that we feel that grades 13 through 
16 are not important. It simply means that we had a vast problem in front of us, 
that we couldn't do it all, and that we picked out a piece of the job that seemed with-* 
in our powers, hoping that other organizations would come along and fill in the gaps. 
I am happy to report that this is exactly what is taking place. Finally, our deci« 
sion did not involve an evaltiation of htunan worth, but rather an evaluation of math** 
ematical potential. We can no longer neglect our responsibilities for our college** 
capable young people* 

Beginnings are difficult. In designing a new program, you can begin by throwing 
out the old one. Some prefer to handle the problem in this way. (In my opinion, 
this is the method being used at the University of Illinois. And I should like to pay 
tribute to the fine work they are doing. I believe that theirs is <^ longer range 
program than ours.) On the other hand, the thinking of the Commission went some-* 
thing like this: Since much of the traditional mathematics is still important and 
significant, let us strive to develop an improved program in which the new grows 
out of the old in a way that makes reasonable demands on teachers and administra- 
tors. In 'short, let us try for a program that is oriented to the future, involving 
changes that can begin at once gradually and proceed forward rapidly. I believe 
that the Commission's recommendations exemplify such a program - a step for«- 
ward that is both real and realistic. 

How did we work out the details of this program? With a team composed of math-* 
ematicians and teachers of mathematics. There are those in mathemaCtics who 
excel in exposition and commimication, and there are those who discover new math- 
ematics. We do not usually find all of these gifts in one individual. But, in design- 
ing a new program, all of them must be avkilable. Thus, when it comes to deciding 
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whether a certain mathematical idea is relevant, important, obsolete, obsolescent, 
or teachable at a given level, the team provides the best answer. If we are going 
to do effective work in improving our mathematics programs, I believe (and this is 
one of the fundamental tenets of the School Mathematics Study Group) that we must 
have teams of mathematicians and teachers - college people and school people- 
working together. A later development, and one that we all welcome, took place in 
Chicago recently when psychologists, mathematicians, and teachers consulted to lay 
plans for an improved program in elementary school mathematics. 

One of my greatest professional experiences has been that of working with mathemat- 
icians. Teachers and mata^;maticians in the writing teams of the Commissio.i found 
that they needed each other. Each had something to learn and something to contrib- 
ute. At the first meeting of our writing group, we were all somewhat diffident. We 
thought there were going to be misunderstandings and violent differences of opinion. 
But such misgivings proved to have no foundation in fact. Differences of opinion 
were followed by highly constructive discussions leading usually to modifications of 
viewpoint and to meetings of minds. 

I was reminded of the man w lO invented a wonderful button. He claimed, "When 
you press this button of mine, no light goes on, no fan starts whirring, no refriger* 
ator goes into action. No, Sir, when you press this button of mine, it simply presses 
back on you and makes you feel wanted." In our work with the Commission, we 
pressed back on each other, and both parts of the team felt wanted. 

I have talked about the "why" and the "how", Now what are some oi the recommen- 
dations that we make in our Report? What are some of the things that we urge for 
the modification and improvement of high school mathematics for the college -capable? 
Let me make it clear at the outset that the Commission never has believed that there 
is a unique program in mathematics suitable for all schools from Maine to California. 
This would be folly indeed. But we do believe that there are certain ideas that are 
worthy of general consideration. 

The first recommendation to which I should like to call attention urges a thoroughly 
revised estimate of what should constitute a minimum study of mathematics for college- 
capable students. A year of algebra and a year of geometry are shockingly inadevaate. 
We believe that it is the job of the home, the school, and the guidance counselor to 
let talented young people know that many doors are closed to them these days if they 
haven't a solid foundation in high school mathematics. And these are not only doors 
leading to the natural sciences* All too frequently, a boy or girl proceeding to the 
study of social sciences finds little resistance to the dropping of mathematics early 
in high school. This may have most unfortunate consequences. Talented young people, 
we believe, should get a minimum of three years, and preferably four years, of col- 
lege preparatory mathematics, even if they do not plan to study mathematics or the 
physical sciences or engineering in college. They may well find their mathematical 
needs bounded only by the limitations of their talent. 

As for content, the Commission on Mathematics is recommending a program that is 
about 80 per oent traditional. We have been criticized for having too conservative a 
program and for having too radical a program. Any teacher who studies the program 
carefully will realize that most of the material is the same that has been taught. What 
we do recommend is a marked change in the spirit and in the methodology of the 
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teaching* We want to breathe into the program new points of view, in which cer- 
tain ideas and concepts that mathematicians have found to be powerful and impor- 
tant are used to simplify, clarify, and unify much of the traditional material. 

We are not for one moment suggesting that concepts replace skills^ I hear some 
ridiculous^ statements to the effect the Commission has little use for mathemat- 
ical skills. This is nonsense. We have never regarded concepts and skills on an 
"either-or" basis; the Commission on Mathematics regards concepts and skills on 
a "both-and*' basis. We wonl! settle for one without the other. 

What are some of the things that characterize this new spirit? One thing that we 
should like to bring into the teaching of mathematics should begin quite early in 
arithmetic. This is an appreciation of mathematical structure, or as Sawyer has 
called it, ''patterns". The properties of the number system, we belie -e, can be 
reconstructed quite early in the experience of young people to give meaning to 
mathematics. Basic laws support the mathematical structure in the same way 
that steel girders hold up a skyscraper. The full importance of these laws nas 
been brought into the spotlight only within the last 30 to 40 years. Yet these prin- 
ciples ccin and should be discovered quite early. Now, some people have taken 
this to mean that the Commission wcuits to teach abstract algebra in the eighth or 
ninth grade. Not at all. We are not foolish enough to believe that the abstract 
should precede :he concrete, but students can learn more about both abstract and 
concrete thain is commonly supposed. The only thing holding back many of our 
able students is the inadequacy of their teachers, of whom I am one. It is a sober- 
ing thought when I realize that today students in the eighth grade in my school can 
handle inequalities better than I could handle them as a sophomore at college. Two 
fallacies that we teachers must avoid are the notion that if something is new for us, 
it^s impossible for our students; cind the notion .hat if we learned a topic in the four- 
teenth grade, our students must learn it in the fourteenth grade. We must never 
forget that, in dealing with talented students, many may be far more gifted math- 
ematically than we are. Sometimes about the best thing we can do for the very 
talented is to get out of their way. 

But let us return to a consideration of patterns in mathematics. These patterns, 
or structures, pervade mathematics. Y>u find them in arithmetic, and they recur 
in algebra, geometry, and trigonometry. For instance, there is the law that, for 
any number a, a times 1 equals a . Now, that may not seem like very much to you, 
but the implication that you can multiply any number by 1 and leave its value un- 
changed turns out to be a very basic principle. It enables us to perform all sorts 
of transformations and simplifications on fractions, for example. 

Another principle of great importance is symbolized thus: ab = ba. What does 
this important yrt perty of numbers mean? It mesins that if you are multiplying two 
numbers like 3 and 5, the order doesn't make any difference; you can take 3x5 
or 5 X 3. This basic property of the number system has profound implications. 
Most operations in life don't work that way. The order usually makes a difference, 
as in putting on one's shoes and socks. I heard one teacher ask a pupil, "What does 
'ab = ba' mean to you?" xhe youngster replied, "It means the multiplication tables 
are cut in half. " Good for him. He saw an important consequence of the law. He 
didnt have to learn r.fferent results for 3x5 and for 5x3. The foregoing are ex- 
amples of patterns and illustrate what we mean when we talk about an appreciation 
of mathematical structure- the appreciation of the properties of our number systems. 



24 



1 should like now to mention another important point in the Commission's program. 
We recommend the use of ideas that unify the study of mathematics. In particular, 
we believe that the notion of a "set" provides us with a good language with which to 
talk about much of high school mathematics. Contemporary mathematics has given 
us the idea of a set, which is just a collection of objects. You use the idea when you 
talk about a set of dishes, a set of golf clubs, or a set of stamps. This concept of a 
set is pervasive in mathematics. Virtually all the mathematics we know caa be de- 
veloped from the concept of 2 set and the laws of logic. I am not talking about intro- 
ducing complicated set theory in high school. As a teacher, I say this: I can intro- 
duce algebra more effectively by using the idea c.^il language of sets than by any other 
way I have ever devised. It is a magnificent uniiiying concept. 

The Commission has further recommendations. We urge teachers to teach algebra 
so that it emerges as a deductive science, not just a bundle of tricks. There should 
be proofs in algebra, as well as in geometry. And we propose that in geometry we 
face frankly the fact that in the last 200C years we have learned something about the 
subject. We must go beyond Euclid now, repairing defects and removing blemishes 
in » far as is possible and reasonable in high school mathematics. For one thing, 
we can introduce coordinate geometry and make use of our wonderful number system. 
Euclid didn't have a good number system, but we have, and we should use it in geom- 
etry. 

I suppose the most revolutionary thing we are suggesting is that a course in proba- 
bility with statistical applications might be tried in the twelfth grade. (Tliis is just 
one of the possible courses recommended"^) We have enough evidence at hand to say 
that if the teacher Is qualified and if the students are able, such a course offers an 
exciting semester for grade 12. 

Our main objective is to provide for the academically talented a program that will 
prepare them, both in terms of concepts and in ttrms of skills, to tackle calculus. 
Analytic geometry and calculus should be the standard freshman course in this coun- 
try. I think it is a national disgrace that, at the present time only some 10 or 15 
per cent of students who study mathematics as college freshmen are able to study 
calculus. Too many of cur colleges are still offering high school mathematics and 
giv^^ag college credit for it. The sooner the situation is remedied, the better. Some 
ask the Commission on Mathematics, "Why do you not want to introduce calculus in 
the high school?" We have no objection to the introduction of calculus in high school 
if it can be offered as set forth in the Advanced Placement Program. This means a 
solid year of it. But we do not recommend a little dab of calculus. We feel that the 
basic job of the high school is to prepare students for calculus in terms of concepts 
and skills. Moreover, in general, few teachers are prepared to offer a sound course 
in calculus. Much harm may be done by rushing into mathematical responsibilities 
for which due preparation has net been made. 

How do we propose to implement this program? Obviously, notliing is going to happen 
unless somehow or other a great array of teachers can be persuaded to give it a try. 
This presents a major problem for teacher education. All of which gives rise to the 
question: "Is there any interest abroad in the land?" In my opinion, there is terrific 
interest abroad in the land. I can recall nothing to match it in mathematics education. 
Conferences, institutes, and curricular experimentation are the order of the day. 
O , Teachers have written to the College Board for 300, 000 pamphlets and over 20,000 
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copies of the text on probability and statistics. We plan to distribute 60,000 copies 
of the ConunissiDn's Report. Above all, programs for the inservice training of 
teachers are receiving financial support and evoking teacher interest on a scale 
without precedence in this country. 

I don't want to leave you with the impression that the Commission on Mathemat* 
ics of the College Board is the voice of one crying in the wilderness. The Conmiis** 
sion has been discharged. Its members are now working with other groups, such as 
the School Mathematics Study Group, which has the men, the money, and the re 
sources to tackle the problem in a much broader way than the Conmiission has done. 
The School Mathematics Study Group is now busily engaged in a program extending 
from the first grade to the fourteenth. Great things are happening, and I am very 
optimistic about the future. Never in my professi3:>nal experience have so many 
teachers been active in study and interested in moving forward. 

How can teachers prepare to meet the demands of these new programs? Pre- 
service training and inservice training are involved. Of the two, the pre»service 
job is probably the easier. Inservice training is a big job, requiring that teachers 
study as they teach, whether it is self-study, group-study, confer ence-study, or 
institute-study. Wonderful opportunities are being provided for teachers, particu** 
larly by the National Science Foundation. Over a period of years, there will prob- 
ably be enough institutes to accommodate almost every teacher in xxiathematics who 
wishes to attend. 

In addition to the publications of the Conmiission on Mathematics, we shall soon have 
a great wealth of material for teachers in the form of pamphlets, books, and texts. 
As of September of this year, the School Mathematics Study Group will have for ex- 
perimental use a complete series of texts for grades 9 through 12. 

The Maryland project has already tmdertaken wide experimentation with materials 
for grades 7 and 8. These grades, in the opinion of many^ are thin spots in the math- 
ematics program. I am astonished at the ideas that are being put into those programs* 
You have to watch yourself. I've fotmd myself getting into the fallacy of saying, "I 
have never done it. I wasn't taught this way. I don't think it can be done." Then, I 
talk to teachers who are using these new Maryland materials. Almost without excep- 
tion» they report that the materials are exciting to them and exciting to the students. 

I should like to conclude with this thought: These are challenging times for teachers 
of mathematics. Truly wonderfxil things are happening in mathematics education. 
Let no one fail to make his full contribution. It is indeed a time to teach and a time 
to learn. 

Note: The following information is added for those who may wish to obtain materials 
mentioned in the foregoing. For all publications of the Commission on Mathematics, 
write: 

College Entrance Examination Board 
c/o Educational Testing Service 
Box 592, Princeton, New Jersey 

For all publications of the School Mathematics Study Group or of the University of 
Maryland Mathematics Project, write: 
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School of Mathematics Study Group 
Drawer 2502 A, Yale Station 
New Haven, Connecticut 

CHAIRMAN COLADARCI: Are there any questions? Sir, would you identify your- 
self loudly. 

JOHN JOSEPH RISSER (Psychologist, Pasadena City College, Pasadena, California): 
Joe Risser from Pasadena City College. Many colleges experience a very high mor- 
tality in this first course in calculus. I wonder if you can make some comments as to 
criteria or guide posts that can make our counseling more effective. 

MR^ ROURKE: I believe that for effective work in calculus, students need a ground- 
ing of the type set forth in the Commission's recommendations. I do not believe that 
enough students are getting this ki^d of groimding. I think too many of them are sign- 
ing up for courses in calculus when they haven't had the requisite trai n ing in algebra 
and so on, 

A student .is not ready to tackle a course in calculus until he has completed three and 
a half years of the Conmiission^s program as set forth in the Report. In the Report, 
we say that during the period of tremsition, some jschools may not be able to complete 
the full four-year program as recommended; but if they can complete aminimiim of 
three and a half years, to the end of the course in elementary fxmctions, we believe 
that this is good solid training for calculus. 

Of course, there are other things involved besides preparation: goals, gifts, and 
guts. Unfortunately, often social problems arise in college and prevent a youngster 
from doing his best. Even if such a student conserves his energies magnificently, I 
dbn^t care what kind of a foundation he has, he is not going to do well in calculus. 

CHAIRMAN COLiADARCI: Does that answer your question? 

MR^ RISSER: Thank you. 

MIIJJRED ROBECK (Assistant Professor of Education, University of California at 
Santa Barbara, Goleta, California): I have two questions. Teachers of high school 
mathematics ordinarily have a major in this area. With such a background, they have 
had many college courses in content. Question number one: How can the methods 
teachers in one course repeat content and show how concomitantly? Question number 
two: How could content courses at the college level be taught more effectively so that 
content would not be necessary in the methods course? 

MR. ROURKE: I should like to make a comment about your first statement. Did I 
understand you to say that most of the teachers have majors in mathematics? I would 
not concede that. Many of the teachers may have majors, but not in mathematics. 
Moreover, even a major in mathematics does not always guarantee solid training in 
content. I cannot concede your statem^int, if I imderstood you correctly. 

When you ask me how these coxirses in content could be taught, you ask a large ques- 
tion. I cannot give the answer in five easy steps, but I'll tell you this: Get me a bunch 



27 



of teachers together, and I'll show you how. By a "professionalized" course, I 
mean a course in content in which the what, the why, and the how are worked in 
together. When the teacher finishes that course, he knows something about math- 
ematics, its place in the high school curriculum, and its presentation. 

I want to say this in defense of the teachers: A great many of the courses that 
are called "majors" are not mathematics majors at all. Recently I read a de- 
scription of a "mathematics major" course offered by a college in an Eastern 
state. This course was designed to prepare teachers to teach the senior grades 
in high school mathematics. After four years, teachers in the course would not 
know as much about mathematics as my Advanced Placement section in grade 12 
at Kent School. I don't think this is the fault of the teachers. I think it's the 
fault of the professors who designed the course. They thought it was adequate 
for the needs of the Twentieth Century. They should know better. 

ALEX D. ALOIA (Associate Professor of Education, Co-Director of Guidance 
Center, Loyola University, Los Angeles, California); What do we do about the 
youngster who shows great promise mathematically but has no desire to pursue 
mathematics or a science in which he might utilize mathematics? I think we 
have a great many students who show achievement m mathematics on the College 
Board. What can we do to stimulate this in some way? 

MR. ROURKE: I wish I knew the answer to that question. Sir. The Rockefeller 
report talks about ability, character, and motivation* I have talked about gifts, 
goals and guts, the three C's. I have run into this question in every school where 
I have ever taught. It is one of our major problems. My ajiswer to the question 
(insofar as I have an answer) is a belief, an article of faith. In terms of my own 
experience, I believe one way to salvage some of these students is to provide them 
with an exciting and challenging program. Bring them into contact with interest- 
ing ideas. 

But I think there is something beyond this. The home, the school, and the church 
have important roles to play. I think that our students have got to find a faith to 
live by, s'^mething that the students in Moscow have. Communism is a religion to 
them. Tl ey have a doctrine of accoixntability. They feel accoxmtable to the state 
for the full exploitation of their talents. Consequently, it's comparatively diffi- 
cult to lose a talented student in the U.S..S.R. I think the Christian church has an 
answer in the Christian doctrine of accountability. The story of the talents, I think, 
tells the tale. Our gifted people must realize they have some stewardship for their 
gifts. 1 see no other effective answer. 

CHAIRMAN COLADARCI: I think we had better move on. If you have a further ques- 
tion or wish to engage Mr. Rourke further, you will have an opportunity to do it lat- 
er today. I can't resist, however, one parting shot; and then we will dismiss him 
for the day, officially. I was reminded of a story of a citizen who happened to visit 
a high speed computer lab. He was thoroughly awed by the wall full of lights. He 
said to the technician, "I'll bet this machine can do anything." The technician re- 
plied, "It certaunly can. Ask it a question." "Anything?" asked the citizen. "Cer- 
tainly," replied the technician. The man stood off and asked, '^Where is my father 
at this moment?" The lights blinked, and out came a tape saying, "At this moment, 
your father is teaching mathematics at Kent School." The man stood back and said, 
'*Ha-ha, my father, John Randall, Junior, is in his office in San Francisco. " The 
machine blinked its lights again, and out came a tape saying, "John Randall, Junior, 
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is at this moment in his office in San Francisco.. Your father is teaching mathe- 
matics at Kent School*'^ 

Our last speaker for the morning has been put last, I think, only because he is the 
youngest. He will provide a study in contrast because he is a very humble person. 
Dr* Davis is now Professor of Physics at Reed College. He earned his degree in 
physics at the University of Rochester in 1948. His Master's was in physics at 
Syracuse, and his Bachelor's was in physics. He has been a high school teacher 
in physics. So far, you see all our people have been high school teachers. Dr. 
Davis has been science consultajit to the Portland Gifted Child Program and has 
been very active in the National Science Foundation Institute. 

This morning, he would like to describe the origins of the Physical Science Study 
Committee, on which he has been functioning, together with some discussion of the 
work of that committee and some analogous developments in other areas. When he 
is finished, we will have completed the first portion of our program. Right after 
lunch, we will turn to the question of possible implications of psychometry in this 
matter. Dr. Davis! 

RECENT CURRICULUM DEVELOPMENTS IN SCIENCE 
KENNETH E. DAVIS 

The amusing and edifying interaction between two of the persons up here brings to 
mind a remark which is credited to one of the more glamorous in' abitants of this 
city when things weren't going particularly well; namely, "What can you expect of 
4 day that begins in the morning?" This is not to cast any reflections on the kind 
of morning we are having, 

I am sure Mr. Rourke and I owe a good deal of gratitude to the very clear exposition 
of the background that Dr. Tyler gave for the problems we are talking about. 

It seems to be a characteristic of most human beings that only when facing a crisis, 
whether real or imagined, do they ask themselves penetrating and fundamental ques- 
tions. The drowning man whose whole life flashes before his eyes in an instant 
seems to be an enduring part of our folklore. We have seen his analogue several 
times in recent years. A particular awareness and introspection have been common 
in parts of the scientific commimity for some time before the relatively recent events 
which have brought a more general public reaction. This has been alluded to this morn 
ing. Il.has been very carefully pointed out that the scale of these developments is not 
one beginning with Year 1, the year in which words like "Sputnik'', "Explorer**, and 
"space medicine" were the common language of every elementary school student. 

During and inimediately following World War II, some participants in the major pro- 
grams of radar and A-bomb development were struck by a curious fact. If one were 
to list those men whose contributions to such work had been of the most fundamental 
and important sort, not one in the top dozen or so had had his most important train- 
ing in this coimtry* Only a few were native Americans, and their training had been 
received abroad. There was a good deal of conjecture as to the reason for this. It 
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seemed that something was lacking in the scientific training provided in this coun- 
try or that we were inefficient in developing talent. One feels reluctant to conclude 
that our population lacks its proper proportion of brains. The numbers and general 
conditions are such as to make one feel that in this, at least, we should have statis- 
tics on our side. 

One explanation persistently offered was that unusual talent was frustrated and ham-- 
strung by the regimentation of our educational system, up to and including college 
and graduate work. I would like to return to this point later. Individual observers 
offered solutions of wholesale cutting of red tape when exceptional talent should be 
discovered, but the general result seemed to be a settling back with nothing changed 
much after the conferences and debates. 

About five years ago, the scientific community began to stir in concern once more. 
Surveys made of high school enrollment in mathematics and the sciences since 
1900 resulted in graphs that dipped steeply in the more recent years. This is all 
ancient history now, but I am trying to put it in its context. Professional physicists 
and chemists became quite concerned, since this meant to them a drying up of the 
source of manpower in their chosen fields. It seemed to indicate also that fewer and 
fewer non-science majors were having contact with or even elementary understand- 
ing of the sciences. Since legislators and many others concerned with the leader- 
ship of our nation now need an acquaintance with science even more than ever, such 
a drop in enrollment in science coixrses would have other far-reaching effects besides 
reducing the number of scientists and engineers. We cannot afford to have a large 
gap due to lack of understanding or lack of communication grow between major seg- 
ments of our society. Our citizenry generally must be an informed one if it is to make 
wise decisions. Certainly, we do not want to have decisions made in ignorance or have 
them made by a very few individuals for the many. 

With such concerns in mind, groups of chemists and physicists began to meet in 1956 
to discuss the seriousness of the problem and what they as professionals might do. 
Every profession (and every trade, for that matter) has an obligation to its continu- 
ing membership to take an active part in the proper education ajid training of those 
entering the profession. I would like to underline that statement. It is aimed at some 
of my colleagues who have had a stand-off attitude. In addition, the profession has 
the obligation to inform the general public on matters of fact to the best of its ability. 
For many years, the professional subject matter specialists have in large part shirked 
their responsibility toward their neophytes. A change seems well on its way. . 

In 1956, one discussion group finally crystallized into what is now called the Physical 
Science Study Committee. Drawing its membership from industrial as well as uni- 
versity, college, and secondary school physicists, it elected as chairman Professor 
Zacharias of MIT. Because of our national penchant for catch phrases and slogans, 
this has been called the **MIT Program" by some. Many people feel that this should 
be known as a program which came from a wider source than a single institution. 

The Committee members felt that surely the high school student must not know the 
essence of physics as they knew it and that probably the best contribution the Commit- 
tee could make as a professional group would be to try to bring some of this spirit to 
the high school student through short, authoritative, and interesting movies on par- 
ticular topics in physics. This was the original idea, the germ from which the pro- 
grajn started. 
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If one were to use such films to illustrate and supplement an existing high school 
course, the films should be preferably tailored to fit the existing text books. The 
Committee members took as a reading assignment the IZ leading high school phys- 
ics texts. To a man, their conclusions were the same: The books did not tell the 
story of physics, or even of science generally. They found instead that the books 
consisted of approximately 50 per cent devoted to sections on technology, among 
which physical principles were stated in unmotivated, underived form. There was, 
to them, little of the coherence and reasoning which they knew and liked in physics; 
but rather, the physics that was there was fragmentary and catechismlike. They 
concluded that they could not in good conscience attempt to make films that would 
materially aid a course based on such books. They had only one course open if 
they wanted to offer their help: the proposed program would have to be enlarged. 

I would like to straighten out one misconception which may exist. I am not a mem- 
ber of the Physical Science Study Committee. I am a Johnny-come -lately who was 
asked to help in some of their training of high school teachers. 

I would like to enumerate some of the things that the P.S.S*C. then thought would 
be necessary for the properly rounded program. As plans took shape, they envis- 
aged the following: 

(a) A new textbook which would tell the story of science generally and 
physics in particular. 

(b) New laboratory experiments that would involve relatively simple 
equipment where possible, so that the interested student might re- 
produce it or improve upon it at home, also simple enough so that 
the student would not have a "black box" getting between him and 
illustration of a principle or the examination of some physical phe- 
nomenon. The experiments should be well dovetailed with the other 
aspects of the course, and the work would flow easily from class- 
room to laboratory without artificial and rigid scheduling. There 
is another catchword which has been applied to this program, which 
the Committee resents and feels is too naive a summation of what 
they were trying aind what they feel they are beginning to approach. 
This was the common "do-it-yourself laboratory". 

(c) Seventy short movies were planned, some to be supplemental and 
illustrative, some to provide demonstrations involving conditions or 
equipment not generally available. 

(d) Approximately 70 paperbacked collateral references were planned, 
selling for less than a dollar each, and written by authorities in the 
field with the help and advice of experienced teachers. 

In my work with the Portland program for students of exceptional 
endowment, referred to earlier by Dr. Coladarci, I arrived inde- 
pendently at two conclusions also arrived at by the Physical Science 
Study Committee: (1) The existing high school physics texts are 
very heavily burdened with discussions of technological developments 
of the last 40 or 50 years; (2) There is a scarcity of collateral refer- 
ence material that is at once authoritative and yet elementary enough 
to be intelligible to the high school student. The one exception I 
found was the Scientific American, and there, in the subject of phys- 
ics, one finds only about one article per month. 
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Material in the paperbacks proposed by the P^S.S.C* is not to be 
assigned by the teacher, but the "Science Shelf is to be available 
to. any curious or interested student when he wishes to explore in 
his spare time more advanced information than dealt with in the 
text and laboratory or read about the applications and technologi- 
cal developments which will not be dealt with to any great extent 
in the textbook itself. 

(e) A laboratory guide giving some constructional details or sugges- 
tions for experiments, but leaving much room for the ingenuity 
and inventiveness of the student. 

({) A Teacher's Guide based on the best guess as to what helps and 
hints the teacher would need, supplemented as soon as possible by 
accurate feedback from teachers trying the course for the first 
time. 

If you are acquainted with the cost of things like this, you will be overwhelmed as 
I was by the magnitude of the program they are talking about. They propose to 
make some 70 short films. When the cost is computed for the total program, the 
figures are almost astronomic, and it is clear the Committee cannot pass out all 
the materials free of charge indefinitely. The initial funds v/ere secured from the 
Fund for the Advancement of Education, the Sloane Foundation, the Ford Fovmda- 
tion> and the National Science Foundation. 

In the* stimmer of 1957, about 50 college ajid industrial physicists met with an equal 
number of selected high school physics teachers to work on the implementation of 
the plans. In the 1957-58 academic year, eight schools ran the first trial of the 
course, with the teachers at times living on a hand-to-mouth basis as the develop- 
ing text was supplied to them* 

In the summer of 1958, five summer institutes for high school teachers in the NSF 
general program were devoted to the materials of the P.S.S.C. program. The in- 
stitutes were held at Bowdoin College, University of Connecticut, University of 
Minnesota, Oak Ridge Institute, and Reed College. This is where we got into this 
particular prograim. This year, some 350 schools are participating in the first 
large-scale trial of the course. Next summer, 15 institutes in the NSF program 
will devote their programs, all or in part, to the materials of the P.S.S.C* One of 
these institutes will be here in Los Angeles at Immaculate Heart College. 

The ETS has followed the developments of the program closely and has developed, 
with the Committee members, a battery of tests for the course as well as a special 
form of the College Entrance Board Examination for those that take the course. 

I feel that it will be necessary to postpone further details of this particular program 
itself except for a few remarks and answers to qiiestions you may have, since I 
would like to comment about some of the other things that are happening in physics, 
chemistry, and biology* There are, of course, films that grew out of a televised 
program of physics in the Pittsburgh public schools. The teaching job was done by 
one of the most outstanding physics teachers in the country. Dr. Harvey White of 
Berkeley. 

First of all, I have a few general observations as far as scierce teaching in the 
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secondary schools is concerned. The Portland public schools are conducting an 
examination of their curriculum for the college-bound student. This is being done 
by a committee taken from the colleges of the state. They brought in some out- 
side directing ht;lp and some examikiing help and are using staff members of the 
colleges in the state. I happen to be one of six consultants or examiners whose 
attention is confined to the science curriculum. Through participation in this study, 
I have become increasingly aware of something about which I had previously only 
had a few hints in other school systems. There seem^ to be a recognition in some 
school systems that the elementary school pupil is being exposed at home and in 
school to much of the material that has until recently been considered to belong in 
the ninth grade general science class. In some systems, this is leading to the im- 
mediate suggestion of putting the biology course in the ninth grade, chemistry in 
the tenth, and physics in the eleventh. There is a difference of opinion, if you want 
to call it that, as to whether the order should be chemistry first and then physics, 
or physics first and then chemistry. In some schools, it may be either one. There 
has been a suggestion that students strongly science oriented then be offered a sen- 
ior seminar in science, the makeup of which is at present fairly indefinite. A . least, 
there are many different types suggested. 

The physical science course has a somewhat ambiguous status in many schools, and 
its future does not seem clearcut. In some schools, it is offered in the ninth or tenth 
grade to able students not primarily interested in the physical sciences but needing 
a greater spread in their science experience. In other schools, it becomes a repos- 
itory for those students who have been unable to pass their state-required one year 
of science in amy other course. Unfortunately, in some oc the school systems, it be- 
comes a class which has a mixture of the two. This, I think, is a most unfortiinate 
situation. I will return to the idea of the physical science course a little later. 

Concern for the elementary trcuning in their fields and a better articulation between 
the high school and college courses is being shown in chemistry and biology, as it 
has been in physics. 

From J\me 17 to June 28, 1957, the Reed College conference on the teaching of chem- 
istry was held xmder the sponsorship of the American Chemical Society and the Crown 
Zellerbach Corporation. It was attended by 15 high school and 18 college teachers of 
chemistry. Most sections of the country and all sizes of schools were represented. 
The aim of the conference was a better interrelation of the high school and first-year 
college chemistry course. Principal recommendations of the conference were that a 
new ai^d fundajiriental approach be made to the teaching of high school chemistry by 
adopting chemical bonds as the central theme and then an effort be made to prepare 
the materials needed. The conference also concerned itself with teacher training and 
conditions under which the teachers work. The recommendation was made that the 
minimum standard for certification of teachers of high school chemistry be 16 semes- 
ter hours, including organic chemistry, and that the graduate training should empha- 
size subject matter areas in science and mathematics. They felt that high school stu- 
dents should be given teaching opportunities and that the use of student assistants should 
be encouraged through the granting of credit or financial help. A definite course plan 
that developed from this conference was the work of Dr. Laurence Strong of Earlham 
College and Dr. M. K. Wilson of Tufts College, based on the theme of chemical bond- 
ings 
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This conference was followed by one at Wesleyan University in Middletown, Con- 

H necticut, June 16 to 26, 1958, with many of the participants from the Reed confer* 

ence in attendance. This conference worked in three separate groups, only one of 
them being closely associated with the previous conference work. General con« 

I elusions of the conference favored several approaches to the problem of design of 

the high school chemistry course with close collaboration and feedback between the 
programs. They specifically felt that they did not want to commit themselves to a 

I single approach "as the P.S.S.C. has done.'» The third of these conference$, spon- 

sored by the NSF, will be held this coming sximmer for six weeks on the Reed Col- 
lege campus. I'm sorry if I keep referring to Reed. It happens in this particular 

I instance that these conferences were backed by the American Chemical Society, 

and they had the first one, and now the third one, there. 

J Other activities in chemistry, independent of the conferences mentioned, include 

the following: 

T (a) The Mauiufacturing Chemists Association has produced "Science 

Experiments in Chemistry", published in 1958. These consist of 
guide sheets for 31 open-ended experiments requiring students to 
devise their own procedures and draw original conclusions. 

(b) As a result of an advisory conference held at the National Science 
Foundation in December, 1958, there was set up an Interim Planning 
Committee which is actively developing a chemistry curriculum study. 
The Chairman of the Committee is Dr. A. B. Garrett of the Depart- 
ment of Chemistry at Ohio State. They expect to "formulate a definite 
proposal by the end of May". 

(c) A filmed chemistry course has been prepared by J. F. Baxter and 
N. E. Bingham and is scheduled to be issued by the Encyclopedia 
Britannica Films in 1959* Some people have said this is comparable 
in magnitude to the number of films involved in Dr. Whitens program, 
to which I previously referred. 

(d) There are various small groups working. I know of one in Redwood 
City where one of the high school teachers of physics and chemistry 
informed me that he and three other members of their school system 
will be working on the development of a chemistry course in a some- 
what analogous form to the development of the P.S.S.C. physics 
course. They will be subsidized by the school system to work on it 
in the summer. 



In biology, the earliest of the present movements seems to be the one initiated by 
the Committee on Educational Policies, Division of Biology and Agriculture of the 
National Academy of Sciences, National Research Council. In the summer of 1957, 
after general canvassing for original laboratory and field suggestions, a selected 
group of high school and college teachers of biology met for eight weeks at Mich- 
igan State University. In the preliminaries to that meeting, they issued a general 
call for suggestions, original experiments, pieces of information, and so forth. 
From among those people who submitted aterial, they then selected the partici- 
pants. As I recall the remarks of one of the teachers, there were between five and 
ten times more people who submitted material than they had space for at the con- 
ference. 

^ Their goal was to work out laboratory and field exercises according to an outline 
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that had previously been prepared by a panel on high school courses* They produced 
a mimeographed manuscript which was made available to about 100 select high schools 
for trial and evaluation* The material is presently being revised and edited by Dr« C. 

A. Lawson of Michigan State and is scheduled to be published commercially this com* 
ing summer. The Committee on Educational Policies was discontinued as of June 30, 
1958, since its original assignment had been completed. I say this because many of 
the biology and chemistry projects are much longer range than this« 

The American Institute of Biologic Sciences Curriculum Study Steering Committee was 
established late in 1958, holding its first meeting in February, 1959» under the chair- 
manship of Professor Bentley Glass of Johns Hopkins University* The study is to be 
concerned with the teaching of biology in high school, elementary school, and under*- 
graduate college, in that order. A full time director has been secured in Dr. Arnold 

B. Grobman, on leave from ihe University of Florida. His office and staff are located 
at the University of Colorado, Boulder, Colorado. At present, the Steering Committee 
is selecting subcommittees for the detailed carrying out of ^ae program. The previously 
mentioned work of the National Academy Committee has been independent of that of the 
A. I. B .*S. 

It is the feeling of many physicists and chemists that the basic uxiity of science, espe- 
cially in the physical sciences, should be shown in the way the introductory courses are 
taught. The arbitrary dividing lines should not be heavily drawn, and an integrated 
course in the physical sciences would make a great deal of sense. 

At present, the chemistry course requires a significant amount of physics before it 
cajx get started in dealing with the ideas of atoms, molcculea, and their interactions. 
On the other hand, as the P.S.S.C. program shows, one can quite naturally get in<- 
volved in things chemical when discussing atomic or molecular physics or the nature 
of crystals. Some people have been giving serious thought to such a course, develop*- 
ing the important aspects of physics and chemistry together over a two-year period. 
The only serious obstacle is a purely administrative one: what one would do about 
transferring students. 

It should be clear from the list of movements mentioned that there is a joint action 
on the part of high school and college teachers with careful thought being given to 
common problems. Help has come from many private foundations, as well as the Na- 
tional Science Foundation. It should be noted in passing that the NSF is so well aware 
of the particular problem that it has a special division entitled ^'Course Content Im- 
provement Section'' headed by Dr. Arthur S. Roe. This section has provided the nee** 
essary encouragement and support foi many of the newer programs. 

As a result of working with the P.S.S.C* » I feel it is the best thing that has happened 
in the teaching of physics in over 50 years. For the first time, high school and col- 
lege people are working together on our common problems. It seems clear that this 
is true in the other programs as well. It is a good and healthy situation and one long 
overdue. 

I have some of the materials of the P.S.S.C* program with me. If any of you care to 
look at them afterward, I would be glad to show them to you. The collateral references 
which I said were needed so badly are beginni|ig to appear. Three titles are currently 
available. By next summer, there should be about five or six. One is on magnets. 
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with the subtitle > **The Education of a Physicist^S by Dean Frances Bitter of MIT« 
There is one on "Soap Bubbles and the Forces Which Mould Them*\ which is in- 
teresting since it was written before the turn of the century and still stands as a 
good reference. It is a classic and is beautiful material for reference at this lev-* 
el« I think the original publication date was somewhere in the 1880^s« The third 
one is called "The Neutron Story" by Donald Hughes, whom you may recognize as 
one of the authorities in the work on neutron physics. 

The text itself has appeared in four parts, which are referred to as four "vol\unes"# 
I think that the word "volumes" has an appalling soxmd, but it really isn't as long as 
it might sound* There has been continual work of the committee on this coxmt to re- 
duce to a minimxun what is required to tell the story and at the same time to tell the 
story in a coherent way* 

In addition, there is a laboratory guide* I have just a couple of them here* There 
are now four* The teacher's guide is perhaps the one which may startle you* This 
is one of the smaller parts of the teacher ^s guide* (Indicating) This is the part^ 
developed for the first volume* The other two presently in existence are larger 
than this* 

I think the work of the Physical Science Study Committee does reflect careful and 
able work on the part of people who know what they are talking about in a particular 
field of specialization. These are people who are also able enough in the broader 
sense to talk with those who face the problems of teaching this information* I feel 
this is a very exciting program* One is continually impressed by the impact it has 
on the students and the way in which the students seem to get the flavor of what you 
do when you do science. You do not just talk about it nor believe science is a body 
oT facts which are issued by som*^. oracle in a white coat* Thank you* 

CHAIRMAN COLADARCI: Thank you, Dr* Davis* Do we have any questions? 

DR. JOHN C. GOWAN (Associate Professor of Education, San Fernando Valley 
State College, Northridge, California): I think many would like to know the pub- 
lisher^ address* 

DR. DAVIS: There is a very serious problem here, as all of you know, in that the 
adoption of textbooks for a course in most school systems is something that is a 
four or five year business* The problem arises from the fact that this course is 
not one that has been developed. It is one that is still in the process of being_de- 
veloped. As a matter of fact, one of the things I have here is evidence of this* 
(Indicating) This is the first of the four volumes* This other booklet is the re- 
vision of about half of it, which appeared this year • the result of the schools' first 
year of work* 

The answer to your question is that the group has now become incorporated imder 
Educational Services Incorporated, with the P.S*S*C* as a subdivision of this* 
Until the book appears in hard covers (which they expect to be about the fall of I960), 
the material will be essentially in the form I have indicated with the revisions ap- 
pearing in small paper-bound form* One can obtain these materials from the P«S*S.C* 
Their address is 164 Main Street, Watertown, Massachusetts* I do not know what 
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the prices are* I thixik originally they were charging something like two dollars 
for each of these quarters of the text. It is intended that the eventual cost will 
be five or six dollars^ which will be in the same range as most textbooks now. 
There is also the laboratory guide material and the teacher ^s guide. The mate* 
rial in the teacher's guide is available only to those teachers that are taking the 
institutes on this course, 

CHAIRMAN COI4ADARCI: Are there any other questions for Dr. Dav;s or for any 
of the speakers? 

DR. JACK WILSON (Professor of Mathematics, San Francisco State College, San 
Francisco, California): I have a question for Mr. Rourke. You mentioned some<- 
thing about the teaching of patterns in the elementary grades. Many elementary 
teachers seem to have become somewhat fascinated by Dr. Gattegno and his meth<- 
od of teaching arithmetic. Woxild you care to react to this method of teaching 
arithmetic? That is my first question. 

The other question is xmrelated to it. In the publications of the Yale Study Group, 
they are now insisting that the word "equal" be used in the sense of "is identical 
to", whereas the Commission has not favored this approach yet, and thus is not 
using the word "congruent" in the same sense. I wonder ^f you would care to re«* 
act to that. 

MR. ROURKE: I think it would be ve;^ v inappropriate for me to react to yoxir first 
statement about Dr. Gattegno. I have had no experience whatever teaching grades 
one through seven. I think it would be better if you were to ask someone who is an 
expert in this field. 

Next, with reg3trd to the use of "equal" r ' "congruent", neither tne School Math- 
ematics Study Group nor the Commission is insisting on anything. It is ttitie that 
on the surface, there are some differences of opinion on what should be done in 
geometry. The simple fact is that neither of us knows what is best. The Commis* 
sion has taken one stand to modify Euclidean geometry, to repair it where we can, 
and to be honest about it where we can't. The School Mathematics Study Group is a 
little more optimistic about the possibilities of axiomatic synthetic geometry than is 
the Commission. We do not suggest the distinctions that they make in connection 
with the words "equal" and "congruent". But we should have no objection whatever 
if someone finds such distinctions useful in high school geometry. There are many 
roads that lead to Rome. 



(The morning session was recessed at 12:15 p.m.) 
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AFTERNOON SESSION 

The meeting was reconvened at 1:45 o'clock. Chairman Coladarci presiding. 

CHAIRMAN COLADARCI: Between the ending of the morning session and the be- 
ginning of lunch, there were a couple of comments to the effect thixt there hasn^t 
been an unintelligent word spoken so far. These remarks were rather sincerely 
made. In many ways, this is not only an important evaluation of the morning but 
also a prediction of the afternoon. If thr-^ two people let us down, the blame is 
all theirs. 

While I now give them time to think of something to say, I might report a little 
bit of interesting intelligence. It involves one of the ex-key figures in ETS, 
John Caffrey, whom you all know. John is now Director of Reset ch in the Palo 
Alto Schools. He is partial to IBM equipment, as you probably know. One of the 
thinps John has done is to reduce reporting to parents to a punchcard operation. 
Now, at the reporting period, parents in Palo Alto receive a little card with holes 
punsthedin it and various esoteric symbols. The first tirae he tried it out, appar- 
ently, many of the parents were confused. The holes happened to fall in the wrong 
places. The whole point of the story, as a colleague observed at a meeting the 
other day, is that the machines are really taking over. They are being treated as 
human beings. We have, as an example, the letter of instructions which John Caf- 
frey sent out to parents with the last report card. It included the statement: "Pay 
no attention to the holes punched in the card. They interest only the machine." 
If John looks rather strange to you these days, you will know ih^l his social inter- 
action is a little bit different from that of most of us. 

As you can see by the titles, the two people we have this afternoon are administra- 
tors. They are vice-presidents of this organization or of its various relatives. 
This morning we had an administrator also. We know Dr. Tyler is a human being, 
however, so we can forgive him. I note this only to pa .s on another little story. 
Someone, at a recent conference, observed very pompously that George Washing- 
ton CDuld broadjump 23 feet. A participant immediately responded, "That's noth- 
ing at all. Any administrator can sidestep twice that distance." This anecdote, 
however, does not approach the infamy of another report which I really saw recent- 
ly. A report on adninistration had, as its first conclusion: ''The data herein re- 
ported suggests that there are three tvpes of schools: large, middle -si zed, and 
small." With research such as this, we are bound to improve. 

The speakers this afternoon are in a sense trying to relate the ideas suggested 
this morning to the concerns of an organization interested essentially in test devel- 
opment. This is a difficult task because it involves, on the part of each of them, 
an attempt to predict what was said; they had to formulate their remarks before 
they came. Those of you who have attended the-.e meetmgs through the years will 
remember that one of the best chairmen in the ' ^rld is Welty Lefever. Some years 
ago, on a similar occasion, he had to summariz le comments of various speakers 
He said, at that time, that he wished he had the s* J>1 of a particular dinosaur. This 
dinosaur had two brains: one in the cranium and one in the pelvic region. As Welty 
said, the dinosaur could therefore think both a priori and a posteriori at the same 
time. The next two speakers would wish to have such an attribute. 



The first of our two speakers is Mr. Kendrick. His initials are S. A. I have been 
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told by everybody in this organization that I should tell you that the S stands for 
^hildrick. It's supposed to be very funny, but apparently it's not. 

He is College Entrance Exajnination Board Vice-President of Examinations and Re- 
search, as the program indicates. He is from Louisiana^ by origin and he appar- 
ently admits it* Mr. Kendrick was educated most recently at Harvard School of 
Education. Before that, he was at George Peabody; and before that, at Northwest- 
ern State College, Louisiana. Is that correct? Is there such a place? 

Among his other professional activities, he has been CoTxnseling Psychologist at the 
Counseling Center at Vanderbilt, a member of the Harvard Business School admis- 
sions staff, Assistant to the Dean of the Graduate School of Education at Harvard, 
and Director of Publications for the Commission on Teat iier Education of the NEA. 
He has been with the College Board since 1955, as Vice-President, He modestly 
says that he has published "very scattered pieces in journals and magazines", and 
he has been editor of two journals: "The Harvard Education Review", and more 
recently, "The Journal of Teacher Education of the NEA." 

He is going to talk to the point of changes in measurement that seem to follow from 
the changes in curriculum that were suggested this morning. Mr. Kendrick! 

CHANGES IN MEASUREMENT AS A RESULT OF CURRICULUM DEVELOPMENTS 

S,A. KENDRICK 

Considering the problem he had with "Joe", I etm surprised that he can pronounce 
Shildrick. 

I know the morning program has sufficiently established the fact that the curriculum 
of the American school is in a state of tension, I admired Dr. Tyler's paper very 
much. My only thought was that I wish I could believe that this kind of thinking about 
curriculum is likely to be that which determines the curriculum in the secondary school 
during the next ten years. I think there is a possibility that some alternative forces 
will be at work. It is necessary, therefore* for me to review them before I begin to 
talk about my subject, which is "Changes m Measurement as a Result of Curriculum 
Developments", 

I might say that I am going to restrict myself to measurement when it comes from 
outside the school and is at least partially coercive. By that, I mean that the test se- 
lection is not in the hands of the school. I use such a narrow definition for my paper 
because I think this is the area in which the most critical point of articulation of mea- 
surement and curriculum occurs. If you can choose your own tests and your own cur- 
riculum, obviously changes depend upon your own abihty and the availability of instru- 
ments. When you are confronted with something like the College Board, or in New 
York State with the Regents; or scholarship exammationsr National Merit, General 
Motors, California State, you certainly are not in the same position to make the neat 
fit that is theoretically desirable, I think that I have chosen the hard part of this 
assignment. It is this difficult aspect that I shall talk about after I have reviewed 
another way of looking at the current curricular struggle* 
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First, I wish to comment about the words "test" and " examinations" and their 
meanings for various groups. I have had my head almost torn off by various 
people for using the word "test" for an essay. Although I thought these were 
synonyms, I quickly discovered that among certain people in this country, a test 
is a wicked thing etnd am examination is a fine thing. 

I think that there are about three major groups now contending to shape the cur- 
riculum which the secondary school will have beginning some time within the 
next four or five years. To begin with, there is the conservative position, rep- 
resenting exactly the position dominant in the American school during most of its 
history up to euid through the first two decades of this century* Those of the con- 
servative persuasion have never given ground to the reform movement which came 
to dominate the talk about education during the 1920's and I930*s. The conserva- 
tives emphasize that the school's primary mission is the cviltivation of the intel« 
lect. They reject or deplore the school*s custodial function, scoff at adjustment, 
especially at life adjustment, and stand strongly for the conventional disciplines, 
conventionally taught. 

The conservatives went out of fashion (but not out of the schools) under the impact 
of the great depression etnd the nearly simultaneous discovery that the college prep- 
aratory function of the usual high school had become temporarily insignificant when 
the American people made the secondary school the "common" school of the Twen- 
tieth Century. The conservatives were helped out of style by the new educational 
research and by the philosophical work of Dewey and his allies, but I suspect that 
the facts of life had doomed them before an opposing rationale had been completely 
formulated. 

The conservatives like examinations but they do not like tests. They welcome the 
external examiner, but they would prefer that he were not a psychologist; and if he 
reads examinations by machine, they assume that he is not fit to talk to. The new 
psychologists, with th'^; '^comprehensible demonstrations against transfer of train- 
ing and their scientific examinations against the older wisdom, were part of the out- 
ward show of the movement that pushed the conservatives from their dominant po- 
sition in secondary education. The conservatives are a part of the great tradition 
of fine American teaching; but if they prevail, examinations will snuggle in toward 
evaluation of a limited number of sharply and uniformly defined courses of study. 
There are worse plans than this. 

The conservatives are now resurgent. The times seem ripe for them, what with 
the Russians, taxes, admirals, and such. Much more important, I think, is the 
fact that the dynamic force has gone out of the liberal movement, at least for the 
moment. But the conservative resurgence is strangely sputtering. Opposition has 
appeared in surprising places, almost amounting to treachery. Eminent mathemat- 
icians in institutions of impeccable social standing are endorsing outlandish doctrines. 
The endorsements being mainly in mathematical notation, it is possible to know only 
that Euclid is on the point of being denounced. 

A former president of Harvard has joined in the fight for high standards, solid sub- 
jects, and hard intellectual work} but he has also become a champion of the com- 
prehensive high school, called for a senior course in citizenship to be taken by ail, 
and (perhaps unwittingly) taken a position with respect to the study of languages that 
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threatens the existence of Latin in the secondary school. 

The Modern Language Association has xinderstandably set up a proper clamor for 
foreign languages for all - or almost all; but the Association is now dominated by 
those who see language more as a tool than as literature and who come with tapes 
and earphones to mock the language teaching of the past. 

There are other examples, but the main point is that the recti leadership of what 
promised to be a conservative resurgence is coming from those whom I sheill call 
modernistic; and who seem at least as interested in reforming the curriculum as in 
-returning it to the solid subjects. 

The rr odernist position on tests is not full-formed, but it seems certain to be ec- 
lectic and more fe^'Orable than not. The modernist will want examinations as long 
as the item-analysi'? <which he generally understajids ajxd rather approves) is care- 
ful and there is no v^r.ietta against free-response examinations. If he prevails, 
there will be a great c ;al of testing of all kinds, but it will be more a part of the 
machinery and less a great crusade than at some times in the past. There are 
worse plcuxs than this one, too. 

There remain the liberals, grown somewhat older now, but heedless of Shaw's ad-- 
monition that every successful revolution should conclude with the execution of its 
leaders. The liberals remain in titular command, but, as I have said earlier, 
their verve seems weaker. They are not so much in retreat as moving in rather 
wavering circles. The liberals have the grass-roots control that 30 years in office 
brings. They also have the usueil collection of vulnerable places needing defense, 
and the really valid original triumphs have become common-place and therefore 
bring their authors no further credit. Or perhaps the truth is that the liberal move- 
ment which did so much for the elementary school never won the high school at all. 
It shattered the old pattern or presided while the old pattern fell apart, but the lib- 
erals never really got what they wanted in the curriculum of the secondary school, 
ajid they now must defend what they never fathered. 

The liberal likes tests and has always used them. Since he would break down the 
boundaries between traditional disciplines and talk about learning in new terms, he 
will, if he prevails, use a great many tests. These tests will be mainly or entirely 
objective and will be worked closely into the educational process, though not into 
the customary divisions of learning. The liberal rather distrusts uniformity, and 
the external examine^, and will therefore multiply tests even more. For a third 
time, there have been worse ways than this of conducting a system of education. 

Now, you and I and a few others who are not able to be here today are above such 
classifications and partisan division. But our colleagues are not (and they are 
numerous) and will sooner or later determine the curricular issues of our time in 
education. I do not know how matters will be decided - probably, as usual, by some 
accommodation among the three schools of thought; but I do believe that before some 
new uneasy consensus is reached, there is a possibility that significant numbers of 
schools, states, or even regions will adopt equally rational but plainly divergent 
curricula. At that thought, for those who work with national testing programs, as 
the poet has it, ''all roads are exceedingly flat, and all hair stands on end." 
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I am concerned here, as my assignment implies, only with tests which stand in 
some direct relationship to the curriciilum. I am also mainly concerned with tests 
which are at least partially controlled from outside the individual school. I think 
I have explained why. 

The technical problem in this situation is reasonably easy to state. Ideally, a test 
which purports to measure formal learning in school shovild be based directly upon 
the curriculum studied in the particvilar school or upon a curriculum decreed by 
appropriate outside authority and ignored by the school only at its peril. The sec- 
ond of these alternatives, the existence of a coercive national authority, has not 
been possible for a long time in the United States, and I hope and expect that it will 
not become possible. The first alternative, basing the test upon the curriculxim 
voluntarily chosen by the school, is obviously out of the question unless there is a 
reasonably good national consensus with respect to the curriculum. When there is 
such reasonable consensus about at least a substantial central core, local differ- 
ences can be expected to cancel out to the detriment of no one if the test is made 
long enough, samples are broadly in the material within which consensus has been 
reached, and if the test is constajitly scrutinized by persons who are in close touch 
with teaching in the schools. 

The College Board's test in the social studies is an example of testing in an area 
where there is some, but not perfect, consensus. There is a long-standing quarrel 
between the historians who would have the schools teach nothing but history (with 
perhaps some government, if suitably historical) and those who have wanted to 
break down this conventional sequence and teach citizenship, internationalism, geo- 
graphy, anthropology, and a great many other units and topics that do not normally 
occur in conventional history courses. By and large, the historians have had their 
way in independent schools and a few predominantly college preparatory public 
schools. Inmost public schools, the broader social studies have prevailed insofar 
as philosophy and over -all curriculum design are concerned. Still, it has been 
possible to make a national test because a substantial consensus has remained. 

Both camps have agreed that American history is at the heart of high school social 
studies; and none has objected to testing for ability to read charts, graphs, and maps, 
to read social studies materials with comprehension, and for understanding of the 
very largest landmarks in Western history outside the United States, etc. 

Ihere has been grumbling on both sides, but the consensus has held and the test has 
worked rather well. There has been only one major difficulty: As the test has moved 
more and more into the area of reasoning with social studies materials and reading 
comprehension in the social studies, its correlation with the verbal score of the 
Board's Scholastic Aptitude Test has risen. This is not to say that the test there- 
fore has no value (although its usefxilness does diminish as the inter correlations be- 
come very high), but it does mean that the propriety of calling this a test of school 
achievement is no longer entirely clear, especially for a particular school. A sec- 
ond kind of accommodation to diversity among schools is the proliferation of tests. 
If the failure of schools to reach general agreement about thf^ curricxilum involves 
a division into two or three clearly defined and inclusive groups, it is theoreticaJly 
possible to issue a test for each group. I say this is theoretically possible because 
there are only a limited number of such arrangements that any organization caji 
make before commxinications with consumers break down. The Board would call 
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them clientele, but they are still consumers. Still a certain cimount of prolifera- 
tion is possible, and this is the solution used by the College Board at the present 
time. 

You heard Mr. Davis talk about the PSSC situation in physics. We have solved 
this division among schools in the teaching of physics momentarily by using two 
tests. One is adapted to the PSSC course, and one is not, being adapted to con- 
ventional physics. 

We are about to issue listening comprehension tests in modern languages as op- 
tional tests for those schools that teach by the direct method in French, German, 
and Spanish. We have long offered two different tests in Greek for a total of 50 to 
60 students a year, some of whom are in Athens. 

I will diverge for a moment. A few years agc^ we asked a subcommittee of the 
Trustees to study the problem of the Greek examination, which was attracting less 
than 50 candidates a year a.nd going downhill. It seemed to stabilize around 30« 
Ths subcommittee took a year to deliberate and consult people in schools and col- 
leges. They came back to report triumphauitly to the Trustees that not only should 
we not abandon the existing Greek test, but they thought it was perfectly preposter- 
ous to have only one test. There ought to be both demotic and Homeric Greek, 
Therefore, we had two Greek tests the next year. This is one problem, you see, 
of conducting a national program for which "Fair for all'^ is the slogan. The pro- 
liferation of tests, then, is a partial answer, but it is difficult. 

The third solution to the problem of diversity (and one which has been very popu- 
lar with many testing agencies) is that of making tests which seek to measure abil- 
ities that are fundamental to later performance in the subject measured, that can 
be developed by widely varying but equally excellent teaching techniques and mate- 
rials, and that are obviously acceptable to teachers of every philosophy. The trick 
here, of course, is to make good on the words I have used. It usually turns out 
that when you make an excellent test, it is mainly verbal reasoning and reading 
comprehension in special fields. Some of the best tests I have seen are based on 
this rationale, but all of them tend to fall short of the objective, I have already 
commented that intercorrelations tended to be high among such tests if they are 
reliable tests and that therefore there is a great deal of uneconomical use of test- 
ing time. Indeed, xmder the strict definition I have given, this kind of a test is 
more of an evasion than a solution of diversity. I say this without anything at all 
against the tests for certain purposes, such as college admissions, because it may 
be that the measurement of these long term outcomes of school experience is a 
great deal more importaint than measuring the outcomes of particular courses. 
You must be very certain you know what you are doing when you take this particu- 
lar route to solving a problem of diversity. 

A fourth method of dealing with increasing diversity in the particular case of col- 
lege admissions testing is the proliferation not merely of tests but of testing pro- 
grams and college entrance requirements. This is no solution at all, of course, 
but merely a warning that if the problem of diversity is not soluble, we might find 
that colleges will once again lay down very specific requirements with regard to 
the high school curricvilum and will enforce these requirements with tests designed 
to meet college demands rather than school practices. In a sense, this ctmounts 
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to the college saying that by cutting itself off from a great part of the popvilation, 
it can still get what it wants. Therefore, it does so, I don't think this will hap- 
pen. It would be a calamity to have it happen, and I think we all have better sense; 
but there will be tendencies toward more specific requirements and less flexibil- 
ity in colleges as long as the current debate about the curricvilxxm continues to 
coincide with a seller's market in college admissions. Just last week, one major 
Eastern imiversity announced that, beginning in 1962, admission will be restrict- 
ed to students offering at least three years of high school study in a single for- 
eign langoiage There will be other such annoxmcements. From this position, it 
is but a step to specifying that competence in listening comprehension, rather 
than merely in reading the language, must be demonstrated on a test. The num- 
ber of possible elaborations is immense. 

So far I have talked mainly of the solutions available and the difficulties involved 
in each. Now let me say briefly what has happened and what I expect to see happen. 

Very little has actually happened in national programs for testing school achieve- 
m.ent. Since World War II, there have been some major improvements and impor- 
tant rethinking of tests offered to schools. I would be an ungrateful guest if I did 
not say that the STEP series of the Cooperative Test Division is the outstanding 
example. These do not seem to me to have resulted from reactions to curricxilar 
change so much as from a very useful re -analysis of measurement needs that have 
existed for a long time. 

In the College Board program, we have added tests that I mentioned earlier and ' 
have made what we believe to be improvements in existing tests, but again not in 
response to particular changes in the curricvilum, except for the additions to pro- 
grams. We have added two questions on modern mathematics to the total of 50 
questions in the higher level of our two achievement tests in mathematics. This 
is not enough to affect any student's score significajitly, but we would not be sorry 
to hear that it attracted the attention of a few teachers ajid administrators. We 
have experimented interminably with the English test. This is not because of 
changes in the curriculum but rather because a curriculum in English does not 
exist. 

The future seems a good deal more exciting, if less certain. First of all, the 
gross splintering of the curriculum that I postulated earlier may not happen. 
Rather, schools will have somewhat less freedom to go their own way than they 
have had before. I doubt that the tighter controls will come from testing agencies 
except as a secondary response to the things schools themselves are doing. The 
controls are more apt to come from within the schools themselves or, if not, from 
the public and the colleges. 

Actually, I see all too little rather than too much resistance to the dominant trend 
at the present time* It is becoming increasingly easy to solve public relations 
problems by hasty adjustments of the curriculum to implement the Conant Report, 
or the recommendations of the College Board Commission on Mat,hematics, or , 
some other especially noticeable recommendation for change. Almost any super- 
intendent can buy a semester or so of relative peace by finding someone to speak 
French to the fourth grade for 20 minutes a day*, I deplore not the teaching of 
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French in the fo\xrth grade but the fact that some of us are going along with the tide 
very easily. In any event, I expect somewhat greater uniformity in school practices 
to reduce some of the problems that coxdd result from a period of curricular reform. 

Secondly, there will be even more tests, including, I am afraid, tests thrust upon 
the schools by outside agencies under circumstances not within the schools' control* 
As one such outside agency, the College Board is greatly aware of the growing bur** 
dens imposed upon schools. We hope to control our own contribution to this burden; 
but if the curriculxim fragments, the pressure for new tests and therefore new test- 
ing occasions will increase. 

Finally, if schools do adopt many very different kinds of programs or if colleges re- 
gress to the kinds of entrance requirements that were familiar in the 1920 *s or the 
kinds of testing that were conmion even earlier, the clear solution for national pro- 
grams will be to abandon end-of<*course achievement testing and develop the kind of 
comprehensive half-way house between aptitude and achievement which measures 
the long-term fundamental outcomes of learning over many years but does not at- 
tempt to fit itself to the specific aims of short-term^instruction. In fact, the College 
Board at least is making such a test to shoyc. around in 1959-60, merely as a subject 
for conversation. In spite of my earlier remarks about such tests, this will not be 
a bad solution if we have to come to it. 

I have one final word which has nothing to do with measurement, perhaps, or with 
tests that are in development or in existence. I think that however things come out, 
the strain upon the expertness of g\iidance workers and testing people generally is 
going to be enormous. As the curricula become more complicated, the more com- 
plex will be the college admissions requirements, and the anxiety of both students 
and parents will mount about these outside testing programs, and the more knowl- 
edge guidance and testing people and instructors are going to need« 

Your chairman mentioned this morning the man who didn't want to interpose the 
test between himself and the child. Well, I know his older brother. He is the fellow 
who is willing to accept the test but learned long ago that you have to interpret it 
with caution. For a long time, I felt that by interpreting with caution, people meant 
what I mean when I say you must interpret tests in context. I learned that this isn't 
true. The term too often means that since you don't really know too much about the 
testing business, when a decision comes up there are two possible courses of action; 
(1) You cannot make the decision and let sometixing happen beyond your control; (2) 
You can make the decision, go home, and lose sleep all night. This is being cau- 
tious. This isn't what we want for the children that must filnd their way through the 
possible maze of testing, requirements, and curricular change that the next ten 
years could bring and, I think, -will bring. 

CHAIRMAN COLADARCI: Thank you, Mr. Kendrick. 

Robert L. Ebel is Vice-President of Testing Programs and Services at ETS. He 
has responsibility for test development, administration, analysis, and direction of 
programs, a role he has held since 1957. He was at the State University of Iowa, 
where he was Professor of Education, Director of the University Examination Serv- 
^ ice, and also Director of the Bureau of Education Research. There is an old 
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Japanese saying which translates freely into something like this: "A frog at the 
bottom of a well cannot know the ocean." Dr. Ebel denies the validity of that 
generalization, because he was born and raised at the bottom of the well. I see 
he was born in Iowa and went to public schools there. You know what Iowa is. 
It^s the place where all tests are standardized. He took his B^A, at the Iowa 
State Teachers College, his M.Aw at the State University of Iowa, and his Ph.D. 
at the State University of Iowa. ETS rescued him from that state in 1957. 

r Despite this impoverished experience, he has amassed considerable breadth, 
learning, and intelligence. The Japanese are going to be disappointed to learn 
that one of their propositions does not have as much validity as they thought it 
1 might. He is a member of the NEA, the American Psychological Association, 

■ the American Association for the Advancement of Science, and the Psychomet- 

ric Society. He is Past President of the National Coxmcil on Measurements Used 
I in Education and a member of the Executive Committee of the American Educa- 

tional Research Association. He is author, as I am sure you know, of several 
tests, and a prolific contributor to the research literature on test construction. 

He is going to speak on "Judging the Usefxxlness of a Test in Curricxilum Meas- 
urement". I had a chance to read liis paper a few days ago. Although I \mder- 

J stand he has made some changes in it since, I was quite impressed with it. 

Dr. Ebel will assume the aggressive stance which Mr. Kendrick has already 

_ taken. What he is going to say he really means; and I imderstand from his friends 

that when he really meauis something, he says it rather strongly. Dr. Ebelf 

JUDGING THE USEFULNESS OF A TEST IN CURRICULUM MEASUREMENT 

ROBERT L. EBEL 

I am sure that by this time you all appreciate the problems of a speaker follow- 
ing an introduction by Professor Coladarci. I hadn^t gotten well acquainted with 
him until last evening. But as soon as I did, I began to sense what the problems 
might be and took counsel with myself as to how I shoiild proceed. I realized 
very soon that there woiild be no point in trying to trade quips with him. He is 
far too bright for that. Since I was last on the program, it occurred to me that 
I might try kindness. I have done so assiduously, and you see the resxxlts. I 
didn^t even have to take any special measures to get his attention. 

There is one thing I have noticed about Professor Coladarci that may have es- 
caped you. He is essentially a modest man. In fact, he likes being modest so 
much that he woxild see everyone else in this state of grace if it were possible. 
He has developed some very effective techniques to xmstuff the shirts and let 
the air out of inflated egos. 

I think I can illustrate his essential modesty by relating a story which he will 
deny. Pay no attention to the denial. A short while back, he was delivering a 
speech and was the victim of a very effusive introduction. As a matter of fact, 
the introducer woimd up by saying, "Ladies and gentlemen, I give you one of the 
world^s trxxly great educators.'* Exhibiting this trait of essential modesty. 
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Professor Coladarci's first remarks to the audience were to deny this and say that 
he was not really a great educator but just a teacher and an ordinary individual. 

His speech went very well. The audience responded graciously, and he left the 
hall with his wife feeling euphoric. The statement that the chairman had made in 
introducing him stuck in his mind. Half absentmindedly, he said to his wife, ''Dear, 
how many trxily great educators do you suppose there are in the world?" She was 
silent for a moment and said, "Arthur, I really don't know. But I suspect it may 
be one less than you think." 

Reference has been made to the problems of commxmication that the speakers on 
this program might have. There was a brief quotation from a poem about the di«- 
nosaur and his two brains. This reminded me of another poem \»'hich may be appro- 
priate here. 

I had a little dachshxmd once. 

So long he had no notion 

Of the time it took his head to notify 

His tail of its emotion. 

And thus it was that when his head 

Was hanging low in sadness, 

His little tail kept wagging on 

Because of previous gladness. 

If my comments this afternoon seem out of harmony with the wisdom that was ex- 
pressed here this morning about curriculum, please remember that this panel had 
communication problems too. 

I am delighted to have this opportxmity to talk to you about the quality of tests and 
the effective use of tests. I shoiild make it clear that I am not attempting to pre- 
sent any ETS point of view. I doubt if there is an ETS p oint of view on this subject. 
If there is, I haven't discovered it. We all are working toward the same goals, but 
we think we see different paths toward them. I would get a very vigorous arg\iment 
from some of my colleagues on certain positions that I propose to take. 

Let me begin by referring back to the topic and elaborating on it a little. The topic 
is "Judging the Usefulness of a Test in Curriculxim Measurement". There are two 
general questions which need to be asked in judging the usefulness of an educational 
test. The first is: To what extent does the test reveal the achievement of impor- 
tant educational objectives? The second is: How accxirately does the test distin- 
guish between students having different degrees of this achievement? 

The first question relates to the relevance of the tasks included in the tests. The 
second relates to the reliability of the scores. Together they provide a basis for 
judging the validity of the test. It is generally agreed that validity is the most im- 
portant characteristic of a test. 

To judge the relevance of the tasks in a test, one must first determine what the 
important educational objectives aare. Thenjone must decide how the extent of 
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there are non-intellective components in college performance. We have invited 
people to show us research proposals which we might support* We have gotten a 
large number of proposals. The last group of them went something like this: '1 
don't have much else to do next year* Why don't I give some tests and then rum- 
mage around for a factor analysis?'' We have not supported those. We have got- 
ten some fairly interesting proposals under way. Very few of them are complete. 

Pace and Stern at Syracuse have taken the Murray "need«*press'* approach to in«- 
dividual personality and turned it aroimd upon the institution. They developed a 
college activity index which attempts to demonstrate what the college environ-- 
mem' does and how it fits with the needs of the individual.. Their first effort with 
this was very impressive, so impressive that someone else gave them more money 
to continue this work. 

Thib is one aspect of the thing. It is the non --intellective variable in the environ*- 
ment instead of in the individiial. 

Anastasi at Fordham is still working on a biographical inventory. I know that 
biographical inventories have been tried time and again. She is making an all«»out 
attempt with some very promising data. If nothing comes of it, we cax? at least 
say that a top person with lots of data did it. I think there is a chance we might 
find something out there. 

We have a number of other projects of this sort in the works. Only one test is in 
sight; that is one by Professor Schlesser at Colgate, who has published in various 
places about a test that he has constructed on a questionnaire -type form which, 
astonishingly enough, keeps giving good results in that it doesn't correlate much 
with scholastic aptitude tests and is on a par with college grades. He sent data to 
our research committee, a distinguished body of people. Their comment was, 
"This is astonishing. However, one must either question his integrity, his arith-- 
metic, or suggest that perhaps he has something." We have a lot of money tied 
up in it. I am still skeptical, but not of either of the first two factors I mentioned. 

We rim across the fact that to evaluate non-intellective matters in the adolescent 
personality or whatever, you most frequently have to ask him to be candid about 
himself. Any yoxmgster who would be candid to a college admissions officer when 
he is trying to get in probably isn't ii^tellectually capable of making the grade. 

CHAIRMAN COLADARCI: I think this is the time to draw this to a doae. It's al- 
ways good to be able to say of a conference that we had to stop the questions. 

I'd like to make a few comments « First, I want to say that I sincerely believe 
that today's experience is a very unusxial kind of experience in professional edu- 
cation. We have had five extremely competent people with something to say and 
a willingness to say it. This doesn't always happen. There is usually one in the 
crowd who came to lunch. I was really quite impressed. I feel honored to have 
let my modesty interrelate the five. 

There is one more thing which needs saying. Each one of these persons has pre- 



57 



about students who arv^ over-achievers or under -achiever s> using only the informa- 
tion that >rmerly went into the achievement quotients « Although we can talk about 
the concept of educational potential^ caution and so\il-searching are most appro*^ 
priate when one is entertaining the concept* In short, I do not th?.nk that educational 
potential is an entity which can be precisely measured at this time« 

MR. DAVID KEIRSEY (District Psychologist, Covina School District, Covina, Cal- 
ifornia): I would like to ask Dr. Ebel a question* If I have not fully formxilated it, 
I hope your intuition will tell you what I am asking* In \mdercutting and even drop- 
ping the anchor of prediction, addressing the question of description, and then keep- 
ing in mind Guilford's matrix, what progress has been made in developing tests of 
ability or achievement which specifically and consciously attempt to describe what 
the child is able to do, not trying to find out what he will be able to do? 

DR^ EBEL: I think most of our progress in testing has been in the development of 
tests which describe what an individual is able to do here and now. By looking at 
the educational achievement test, you can form a pretty good picture of what it is 
that the student is being asked to do and what a high score means* 

You can get into all sorts of trouble when you ask the question, "But what does this 
test really measure? The student is answering a question on a reading paragraph, 
but what does this really measure? Is it really a measure of xmder standing, or is 
it really a measure of appreciation, or is it really a measure of something else?" 
We have invented all of these words without defining any of them very precisely* 
I think we would be much farther ahead if we stopped worrying about these hypothet- 
ical traits that are supposed to determine the measure and take pretty much at face 
value the tasks that we set before the youngster « 

This is qtdte at variance with the factor analytic approach to the analysis of human 
traits and abilities. The factor analyst doesn^t undertake to tell by locking at the 
test questions what he is measuring. Instead, he studies the correlations among 
lots of different kinds of tests and looks for common causative agents or factors* 
I would like to point out that you still are faced with the problem of ^amJng the fac- 
tors after they are discovered. To do this, you must look to see Wi:iat the test seems 
to require the student to do* 

In answer to the part of your question about how much progress we have made. J 
would say that we have made enough progress to be proud of it but not enough so that 
we can quit working* '^here is still a lot to be done in measuring current status* 
This seems to me to be very important* 

DR. JOHN C. GOWAN (Associate Professor of Education, San Fernando Valley 
State College, Northridge, California): The discrepancy between College Board 
exams and later performance in college, when it occurs, has often been ascribed 
to non-intellective personality factors* I imderstand the College Beard people and 
the ETS people are working on this* Can you give us some description of what has 
come out of this study? 

DR. KENDRICK: I am speaking for the College Board now. Some years ago, y^e 
annoimced that we were interested in this* It seems reasonable to expect that 
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The second part of the question concerns what the xiltimate psychological effect upon 
the child will be as opposed to the waste of his educational opporttmities* There I 
am skeptical • I know of no evidence and certainly cannot believe that repeatedly 
taking tests (unless they are terribly bad tests) can have any ultimate effect that we 
woxildnH want. Actually, most students enjoy testing if the test is not completely 
absxxrd* 

DR. EBEL; I might add just a brief comment* I agree with what Sam has said, 
particularly the second part. I do not disagree with the first part, but I am partic** 
ularly concerned with this matter of what we might be doing somehow to the child's 
education or to his psyche by testing him all the time. 

The children that I have observed going through this testing process take a fairly 
realistic and balanced attitude toward it. I am very much impressed by how much 
they seem to learn simply as a result of having taken the test. One of my colleagues 
at the University cf Iowa, Professor James Stroud, whose book on educational psy- 
chology some of you may know, has made the assertion that minute for minute, the 
time a student takes in working on a well*»constructed educational achievement test 
is probably the. most valuable educational time he spends. Those of you who watch 
students working on tests that mean something to them realize how much their to- 
tal attention is concerned with the task at hand. If it is a good test, you realize, too, 
how much practice they are getting in the utUization of knowledge. If it is a bad test, 
obviously, it is harmfid. Btit if it is well constructed, it falls in the category, it 
seems to me, of an extremely usefixi experience. It can be an extremely useful 
exercise that actually develops education rather than interfering with it. 

MR. ALEX ALOIA (Associate Professor of Education, Director of Gtiidance 
Center, Loyola University, Los Angeles, California): Dr. Ebel used the word 
*'potential*\ I wonder if we are right in this interpretation of the tests, saying that 
a student has an intellectual potential or an educational potential. What do they 
have when we get through with most of our testing? I know I use the word '^poten- 
tial" until I am blue in the face. Are we simply saying that the student performed 
well in behavior on this particxilar item? Woixld you conmient on that particxilar 
line? 

DR,w EBEL: I perhaps tripped in using the word "potential" because it is not a very 
important element in my conceptxxal scheme. I am much more interested in the in- 
terpretation of a test score which says "This is how he performed on this date on 
these particular tests." What we know about the actual achievement of individuals 
does not bear out, it seems to me, the notion that we can determine the ceiling on 
a child^s potential achievement by some sort of testing. This can get another rip- 
roaring argument started, I am sure. It has to do with the idea of over- and xmder- 
achievement, and the notion of achievement quotients. To determine achievement 
quotients, a student is given one test in which he has to work a number of problems 
and solve verbal analogies so that we can predict how he will perform on an achieve- 
ment test that also involves a nimiber of problems, the use of words, and things of 
that sort. This idea was pretty thoroughly discredited as it applies to achieve- 
ment quotients, but the same notion keeps creeping in the back door when we talk 
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a development of the individxial* the less concerned you are with prediction and 
the more concerned with obtaining a description of what the individual is here and 
now as a basis for determining what future steps ought to be taken* 

I see situations in which prediction is properly the primary concern in the use of 
tests; but I see many more situations in which the validity of our tests ought to be 
determined irrespective of what they predict^ where it seems to me that predic* 
tion is not the relevant consideration* Indeed, even in college admissionsy it is 
possible to take the position that what we ought to be attempting is not to pick out 
those studer.;:: who are going to do best in the curriculum as it now exists but to 
pick out individuals whose educational development has proceeded to the point that 
makes them good college material* Then, if they don't succeed in college, you do 
not blame the tests necessarily* You can take a look at what is happening to the 
students in college* You will look to see if there is some way that this good mate** 
rial is being wasted* 

I think I would stxm it up by saying that prediction, as an aspect of psychological 
measurement, has been considerably overemphasized to the neglect of some of 
these other aspects* 

CHAIRMAN.COLADARCI: Thank you sir* 

MR. DAVID YOUNG (Director of Studies, Happy Valley School, Ojai, California): 
I wotild like to ask what happens to the student who is always being tested? Per- 
haps he is an average student* Are you discovering anything about the students 
who are forever bei^ig tested? I mean, what is happening to his education? Ithink 
Bernard Shaw onc^ bragged that he had never taken an examination in his life* The 
opposite, perhaps^ is happening to some of our students* As it was threatened 
from the platform, we are going to have more and more tests* I wonder what the 
outcome of all this is going to be* 

CHAIRMAN COLADARCI: Whom are you challenging? 

DR^ KENDRICK: I'd be glad to conunent on this* To the extent that all this test- 
ing duplicates itself, it wastes school time and is becoming more and more a se-* 
rious waste of school time* I am frequently asked why the College Board, for 
example, doesn4 get together with somebody or other and agree to have only one 
test* The only possible answer is that after having tried to do this repeatedly with 
various school groups, we usually rim into the fact that the only way to get one 
test is for somebody to go out of business* Our trustees aren't ready to, and 
other people's trustees arenH ready to* 

I think the clamor that the schools are very properly setting up is reaching the ears 
of people* There are some cases where schools can control these matters* ochools 
in nuiny areas control quite effectively the old habit of every firm in town wanting 
to conduct an essay contest and giving out a ten dollar prize* I remember when 
this sort of thing was rampant, and I remember some school principals that han- 
dled it* Many of the pressures on the irdividual schools cannot be handled right 
now, and there is going to be a lot of wasted time« 
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The results are beginning to come in* These students have moved on to senior high 
school. We have a few tear- jerking cases, a few dramatic cases. There was a boy 
who had a measured IQ of 72 who is now a senior in high school with a good strong 
academic record in a college preparatory program in spite of the fact that when he 
goes home at night, his father almost invariably throws his books out the window. 

There is a real area for work here. As far as I am concerned, there is a great 
deal that can be done. If the schools are any good at all, the o^ily place where there 
should be xmdiscovered talent in the junior high school years is in the slums. There 
may be a lot of it there. No one knows. 

CHAIRMAN COLADARCI: Does anyone want to ask Dr. Kendrick to elaborate on the 
distinction between a Broadway play and -z Princeton football game? It is an item 
from the Wechsler-Bellevue, I think. 

Are there any other questions? I am rather surprised that you wotdd let Dr. Ebel 
get away with his rather cursory dismissal of prediction as a useftil concept in talk- 
ing about criterion problems. 

DR. EBELr Would you like me to speak about that? 

CHAIRMAN COLADARCI: Yes. Before you do, may I ask the question? There^s 
one thing about ETS people « they can answer a question before it is asked. Then, 
they deny the relevancy of the question you would have asked. 

You will recall Dr, Ebel asserted that essentially the criterion question for any 
test is a prediction question. Indeed, some people define the meaning of a test by 
what it wUI pre(iict. What is a test for? Let*s find out what kind of things it pre- 
dicts. Cronbach's definition of a validity comes close to this conception. A lot of 
people dismiss this rather quickly, I think, and I would like to hear you speak a 
little bit more about it, if you will. 

DR. EBEL: 1*11 be glad to, Art. I certainly didnH mean to dismiss the idea of 
prediction in all situations. What intended to do was to try to undercut it a little 
bit as the only way of determining the validity of a test. I think there are other 
methods for many educational tests, including achievement tests, and in some 
cases aptitude tests, which are considerably superior. Here one comes up against 
a basic difference in point of view towards educational problems. We probably 
need more research on this, as we do on most of our problems. If we were only 
smar*^ enough, we could discover the extent to which an individual's success and 
his educational potential were somehow built into him as a very young child. We 
might discover the extent to which his educational progress is a predetermined 
imfolding. Now, the notion of boiilt-in., specific aptitudeis the notion of the child 
as a highly adaptable, flexible individual who can be guided and trained in various 
ways. The more emphasis you place on what has been irremedially built into the 
child and on what is there and has to be accepted, the more likely you are to be 
concerned with selection purely on the basis of predi.ction. The more you are con- 
cerned with education as a process of (forgive me for using the word) adjustment, 
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some of the achievement tests, there was a mean gain, for a fair-sized group, of 
about 120 points. The 50 points doesn't worry us too much because of what you 
have pointed out. Certainly, the 120 -point difference can bring into our school 
some boys who we later feel shotjdd not be there. 

DR^ KENDRICK: I don't know enough. No one knows enough about the effect of 
a full year on achievement tests. It may be that things that are not learned are 
reflected on the test. Your 50 points on the aptitude test will happen by growth 
alone a good part of the time. 

DR. JOHN CAFFREY (Director of Research, Palo Alto Unified School District, 
Palo Alto, California): I wotjdd like to ask Dr. Kendrick about the Board's pres- 
ent thinking on the idea of making an attempt to select pupils for college earlier 
than the twelfth year of high school. What has the progress been, if any, along 
that line ? 

DR. KENDRICK: I'm not sure I know, John, whether you are talking about early 
admission. 

DR. CAFFREY: An attempt to select earlier in the school years those pupils 
who should be guided towards college. 

DR. KENDRICK: Well, I don't think we have any novel insight into it. I have al- 
ways taken the position that there are a number of ways in which talent is selected. 
If you are talking about students from middle class homes, if the students remain 
in school for six* seven, or eight years under constant instruction and observation, 
and if at the end of that time the schod doesn^t have a pretty good idea about abil- 
ity to perform, there is something wrong with the school. Th- s is especially true 
if the school uses the conventional tests that are on the market at reasonable prices 
to supplement all of the thousands- of hours of observation. 

There is one particular problem in which I have been very interested. This is the 
deprived child in the lower class. The Board has been putting a modest amoxmt of 
money into a project in New York City where we picked up a bimch of kids from 
Harlem who were of mixed racial origin, including a ntunber of Puerto Ricans 
who had recently come to this country. We made the assumption that testing them 
had not revealed anything much about what they might do if they got the proper 
amounts of instruction, guidance, and help. 

We poured in guidance services. In some of your bcUer school systems, you 
woiildn^t think we poured them in. In the junior high school, where we began with 
a ratio, of 2,000 students to one cotmselor, we reduced it to the ratio of 350 to 
one. We put in a great many remedial reading teachers, remedial arithmetic 
teachers, and we put a social worker on the case. We reduced class size; and 
we took them to everything from the Metropolitan Opera to Broadwa> plays, the 
Princeton football game, and anything that wotdd get them out of the environment 
in which they were then concentrating. 
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Cadet Registrar, U. S. Air Force Academy, Colorado): I would like to address 
a question to Dr. Kendrick. As a user of tests, I am concerned with a particular 
kind of curriculum that is present. It is probably a problem to you. Certainly, 
it is a problem to us as one of your clients. What, if anything, does the College 
Board try to do to reduce the effect of special prepping? We believe from data 
we have and that which we have seen from other schools, that some of the useful- 
ness of the College Board tests in predicting college success has been reduced 
greatly by the so-called prep schools which have been teaching essentially for the 
exam. 

DR. KENDRICK: We have been concerned with this for five years at least. We 
have conducted three intensive attempts to coach students for the Scholastic Apti- 
tude Test. We have invariably failed, although we have tried with more intensity 
than is usual in any coaching school, with the possible exception of some military 
academies with a full year of prep schooling where one gets into the logical prob- 
lem of whether a year of coaching or maturity makes the difference. 

People don't believe this. We keep telling them and showing them that every time 
anyone has data, we are glad to try to cuialyze them with the particular institution. 
We invited a New York City school principal who wouldn't believe it to do some- 
thing with his own people. He divided his jimior and senior high school class into 
three groups: one went to commercial coaching schools in the city, one was coached 
at the school by the faculty, and one had nothing at all. It was qtiite carefully 
checked. At the end of the year, the gains on the Scholastic Aptitude Test were 
identical in the three groups - the means were identical. There were three cases 
in the whole student body of gains of 100 points on the test. One was in the com- 
mercially coached group, one in the school coached group, and one in the group 
that had nothing at all. 

We have done this four times and are perfectly prepared to stand on the fact that 
no method we are able to devise or discover abroad is changing scholastic apti- 
tude tests to a degree which will make differences in college admittance. There 
are some niceties here. If a student is not taking mathematics in the senior year, 
he might get 25 points on a 600-point scale by coaching. He will probably get it if 
he reviews seventh grade arithmetic. 

When you get into the achievement teste, things are different. If you coach for an 
achievement test, what do you do? I assxmie you try to teach the subject. You do 
or you don't. There are logical problems. Tk" essential issue is, "Is rote drill 
on test items the best way to teach mathematics?" I think it is not. If it is not, 
the time for coaching would be more wisely used with a better pedagogical method. 
If it is, good. That's the best answer I can give you. 

CHAIRMAN COLADARCI: Is that respcnsive to your question? 

MR. WESTEN: Yes, I think that's a fine answer. I presimie I may have biased 
data from a group of people who have attended a full year prep course for military 
academy. We found an average gain of close to 50 points on the aptitude tests. On 
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and permit more accurate measurement* This begins to so\md very much like the 
procedizre for building a better test* We began trying to improve tests by im- 
proving criteria and find that the only way to improve criteria is by improving the 
tests* 

The emphasis some test specialists place on prediction, on correlation coeffi-- 
cients, and on empirical statistical procedures reflects a mistrust of himian judg- 
ment, even expert judgment, in determining what a test should measure or in tell- 
ing what it actually does measure* Some of these specialists approach the test 
construction task as if the highest praise for their work would be to say, "He used 
absolutely no judgment at all, either his own or anyone else's in constructing this 
test* " 

No informed or reasonable person would argue that judgment is infallible, or that 
empirical, statistical evidence of validity is irrelevant* But neither should one 
argue that human judgments are completely untrustworthy, and that the only de- 
pendable evidence of test validity is provided by a correlation coefficient between 
test scores and criterion measures* Ultimately we cannot escape the process of 
judgment in determining what we should try to measure or predict* We cazmot es- 
cape the process of judgment in deciding what to call the thing we have measured* 
There is no purely statistical process that can tell us which elements should be 
included and which excluded when a named trait or area of achievement is being 
defined, or what we have measured when the test has been constructed and given* 
Statistical evidence, reduced to correlation coefficients, can help us to avoid the 
jingle fallacy (calling different things by the same name, and henceforth assuming 
that they are the same), or the jangle fallacy (cadling the same thing by different 
names, and henceforth assioning that they are different)* Empirical evidence can 
confirm or refute our judgments bxxt cannot replace them* The judgments of ex- 
perts and the data from experiments are both essential in test validation* 

On the whole, classroom teachers have been far too little concerned with judging 
the usefulness of their tests, either those they buy from test publishers or those 
they create themselves* Much can be done to improve this situation by giving 
greater Pltention to the questions of what is to be zT'^asured, and how it is to be 
measurea, and to an analysis of the quality of the test after it has been given* 
Cooperative test construction, in which each participating instructor must ex- 
amine his own ideas and be prepared to defend them in comparison with alterna- 
tive ideas, is one promising approach to more useful tests* The quality of tests 
a teacher constructs provides revealing evidence as to his competence as a tea- 
cher* A good teacher can give poor tests and weaken his educational influence 
in the process, but a poor teacher, one whose comntiand of subject matter is in- 
adequate, and whose educational values are unsoxmd, caxmot possibly construct a 
good test* Better tests lead to better education* Let us continue our efforts to 
improve the quality of our classroom tests* 

CHAIRMAN COXiADARCI: Well, we have finished the formal presentations in good 
time, so we can consider yotir questions and reactions at leisure* 
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Does the sacrifice cost more than it is worth? In some cases the answer may be 
yes. In others it is probably no. Both judgment and experimental evidence are 
involved in determining the answer. 

It is* well to remember in this connection that any test is inherently and inescapably 
artificial. This is a disadvantage of tests, but it is usually more than offset by the 
advantages of xiniform, controlled conditions for the observation of examinee beha- 
vior and efficient accurate means for recording that behavior. A test using indirect 
measurement is more artificial than one using direct measurement. Hence, the 
differ ence between tests using direct and indirect measurement is one in degree 
more than a difference in kind. When we once decide to give a test, instead of ob- 
serving natural behavior, we have already gone a long way down the road from 
naturalness tc artificiality. To go a few steps further from complex, wordy, prac- 
tical problems io concise, definite questions on basic or incidental knowledge carries 
us a little farther away from direct relevance, but it may yield practical advajitages 
which more than offset the apparent loss in relevance. 

For these reasons, perhaps, we should say, not "A good test of educational achieve- 
ment is one which measxires as directly as possible as many as possible of the ul- 
timate objectives of instruction in a course, " but rather, "A good test of educational 
achievement is one which reveals as effectively as possible the degree to which a 
student has attained as many as possible of the ultimate objectives of instruction in 
the course. In many cases indirect measurement may prove more effective than 
direct. 

In discussing the relevance of the tasks included in the test and the reliability of the 
scores it yields, we have covered the two important components of test validity. If 
a test is composed of highly relevant tasks, if it yields highly reliable scores, then 
it will be a highly valid test. This represents a somewhat different approach to test 
validity than that taken by many test .specialists. It may be worthwhile for us to 
consider briefly the more conventional formulation of the validity problem and some 
of the difficulties it leads to. 

Many test specialists are persuaded that the only proper way to Veilidate scores on a 
test is to determine a validity coefficient between scores on the test and some ap- 
propriate criterion measures. Whenever they discuss the problem of validity seri- 
ously, one of them is sure to suggest that the real source of difficulty is our lack of 
good criteria, and that the Educational Testing Service or some other agency ought 
to get to work immediately to improve criterion measures against which the scores 
of our tests can be validated. This suggestion represents a distinct advance over 
the naive but widely held notion that teachers* nnarks or some other conveniently 
ready-made indices of success, however poorly defined, unreliable, or even irrel- 
evant, constitute perfectly adequate criteria. But still this view reflects a concep- 
tion of the validity problem which is far too narrow, in my view, and which seems, 
again in my view, unlikely to lead to significant improvements in test validity. 

Efforts to improve the estimation of test validity by improving the criterion lead to 
a circle which, if not vicious, is at least frustratingly circular; for, to improve the 
criterion measures, one must define more clearly what is to be measured and con- 
trol the conditions in which it is measured to exclude irrelevant sources of variation 
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problem of how to measure it. A thorough consideration of this topic would open 
the door to discussions of tests versus observations of natxiral behavior, of essay 
tests versus objective tests, of speed tests versus power tests^ of true-false items 
versus multiple -choice items^ and so on. Obviously, we do not have time to con- 
sider all of these now* They all have been discussed extensively elsewhere, and 
while something surely remains to be said about them^ I wo\ild like to direct your 
attention to another issue which has not been so widely considered. In the early 
days of objective testing, short factual test items were in great favor. True- 
false^ completion and matching items were widely used. But attacks on the fac-* 
tual limitations and triviality of many of these questions led to efforts to develop 
more complex item types which appeared to require reasoning as well as recall. 
The pendulum of test construction has kept right on swinging away from concise^ 
factual items until now we see some test items intended to measure higher men^ 
tal processes which seem to be characterized by complexity, verbosity^ and ob- 
scurity or ambiguity. 

For a nimiber of years, teachers have been told that a good test of educational 
achievement measures as directly as possible as many as possible of the ultimate 
objectives of instruction in a coxxrse. I still like the emphasis on laltimate ob- 
jectives and upon measuring the achievement of as many of them as possible. But 
I have begun to have some doubts about the efficiency and necessity of direct meas- 
urement. 

The difference between direct and indirect measures of educational achievement 
needs to be explained at this point. A direct measure requires a student to use 
his knowledge in some practical situation. For example, in a course on educa- 
tional measurement for teachers^ one might ask them to write a good test item^ 
to calculate a standard deviation^ to interpret some item analysis data^ to justify 
a guessing correction formula^ or to deal with a multitude of other realistic prob- 
lems that teachers are likely to face in their use of tests. An indirect measure 
of educational achievement^ on the other hand^ while it also seeks to identify the 
students most likely to deal effectively with practical problems bach as these^ does 
so^ not by presenting the problems themselves, but by testing for basic or even 
for incidental knowledge which the well -trained student is likely to have acquired. 

On the surface, it wooild seem that direct measures of educational achievement 
should be clearly superior to indirect measures. They are designed to be as rel- 
evant as possible, and relevance is what we are seeking. But unfortxmately there 
are complications in the direct approach. Practical problems are often complex. 
It is sometimes difficult to write an adequate answer which will withstand critical 
scrutiny. Efforts to describe the problem situation adequately often result in test 
items that are wordy and time consiuning. The examinee^s reading and test-taking 
skills may become important factors in the score he receives, lowering their val-* 
idity. Because there are fewer scorable responses per hour of testing time, as 
well as because of possible ambiguities in the situations and their descriptions, 
and of possible indeterminancy in the best answers, tests using the direct approach 
often yield scores of low reliability for the amount of time they take. 

The indirect approach sacrificen relevance to gain brevity, clarity and definiteness. 



48 



that we may now have cui excess of curricialar freedom. It is fair to ask whether 
this freedom auid the diversity in course objectives and content which result actu- 
ally give us a better educational product than would be obtained if efforts were made 
to establish greater consistency and uniformity in some common core of essential 
knowledge and skills. The freedom to be different implies the freedom to be worse, 
as well as a freedom to be better. 

If a requirement of greater uniformity in course objectives and content would turn 
inspirational teachers into drill masters, I would oppose it firmly. But I suspect 
that the acceptajice of more xiniform goals and standards might turn some pedago- 
gical dilettantes into more purposefiil teachers. Great curricular uniformity would 
enable us to measure educational achievement more effectively and meaningfully. 
Wisely administered, it could help reward the superior teacher and make the teach- 
ing career a little less attractive and comfortable for one who is incompetent. In 
short, I wonder if the leaders in the educational profession should be quite as ready 
as they seem to be to rush to the barricades in defense of the right of each educa- 
tional institution or of each professor to determine what shall be taught. 

The second point concerning what our tests should measure relates to the impor- 
tance of acquisition of knowledge as an educational objective. The current fashion 
among educators, and even among test specialists appears to be to deprecate, and 
possibly to underestimate, the importance of knowledge. Those who spurn knowl- 
edge as an important goal of education usually turn to some form of mental devel- 
opment, obscurely defined or undefined. '*Teach the students to think, " they urge. 
''Develop their minds." As a htiman being who believes he spends most of his wak- 
ing hours thinking, I stand with those who approve of this function, even though it 
is One in which eggheads and other questionable characters are supposed to excel. 
But I do not quite see bow to separate thinking from knowing. While X am not in 
favor of stuffing anyone's mind with anything, it does seem at times that all of us 
could use a bit more solid, accurate information. 

When the products of my thinidng turn out to be wrong, as sometimes happens, or 
when my efforts to think, produce nothing useful at all, as happens quite often, the 
fault seems usually traceable ro some deficiency in my knowledge. It is possible 
that some malfunction in my cerebral processes may be a contributing cause. But 
just what these processes are and how they ought to operate are still very poorly 
understood even by the specialists. The best way, indeed the only way, I know of 
teaching students how to think is by giving them command over the largest possible 
body of the most useful knowledge. Giving them command over knowledge is not 
the same as teaching them to give back verbatim rote-learned responses. It does 
involve all these things that we have talked about in terms of ability to use knowl- 
edge and understand it. What I am arguing against is the notion that these abilities 
to use knowledge can be developed apart from knowledge itself. There are no ab- 
stract general abilities to think, formulate hypotheses, or to be creative, which 
are not also closely and intimately related to the knowledge that has to go into 
these processeso 

Having considered briefly the disadvantages of extreme diversity of educational 
goals and course contents and the importance of knowledge as an educational out- 
come, let us turn from the problem of wha t to measure to the equally important 
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achievement of these objectives can be revealed. The first problem, Lindquist 
has pointed out, • • is essentially one of curricxilimi construction for which the 
subject-matter expert and the curriculum builder, rather than the test technician 
or classroom teacher, are primarily responsible, • • He continues, "Obviously, 
until the field of achievement in which measurement is to be attempted has been 
specifically and authoritatively described, adequate test construction in that field 
is impossible. It may be noted in this connection that perhaps the most serious 
of the weaknesses which characterize achievement tests now being constructed are 
basically weaknesses of the curriculum rather than of the test techniques them- 
selves." 

If one accepts Lindqmst's analysis, as I do, the ntunber of things a test special! ^t 
is entitled to say about what educational tests shoxild attempt to measure is defin-- 
itely limited. But there are two general points that may be worth brief discussion. 

The first relates to the wide diversity in objectives and content of courses bearing 
the same or closely similar titles. This is particxilarly true at the college level, 
and to some extent also at the high school level. Teachers of the same subject 
have been granted so much freedom to determine what they will teach that some- 
thing approaching anarchy in goals and achievements throughout the country is the 
result. It is not at all imusual, for example, to hear a college professor of English 
literature evaluate a carefxilly prepared literature test in terms like these: '!N[o 
doubt this is a very good test, but of course I coxildn^t use it. It includes many 
things that I don't teach at all, and omits some things that I stress especially." 
The fact that the content of the test in question was selected ajid approved by a 
group of other eminent professors of English literature does not disturb him at all. 
As a creative scholar, he feels not only the right, but indeed the obligation, to be 
\miqae. Professors in many other fields, and even many high school teachers, 
react to carefully prepared and authoritatively approved standardized tests in much 
the same way. 

So strong is the typical teacher *s faith in the value of his freedom, and so great is 
his fear that uniform tests will seriously limit his freedom, th^^ many statewide 
achievement testing programs are maintained only with diffictiltr. Some have not 
survived at all. So strong is the resistance to anything that might limit curricu- 
lar freedom, that the tests used in wide-scale admission and scholarship testing 
programs must be constructed and presented, not as measures of what the cajidi- 
date has learned, but primarily as measures of what he is capable of learning. 
This involves the somewhat surprising assumption that ability to learn is quite 
independent of previous learning, and that the quality of a student^s secondary 
school education has little to do with his performance on the selection tests, or on 
his fitness for college. 

No one should argue, and I am not arguing, that the absence of local curricular 
freedom even in high schools would be a good thing. But I would like to suggest 

* Hawkes, Herbert E. , Lindquist, E.F. , and Mann, C. R. The Construction and 
Use of Achievement Examijiations . Houghton Mifflin, 1936, p. 17. 
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sented a kind of analysis that can be described by the adjectives "rigorous'* and 
''systematic". Each statement was one that had been well thought out. It wasn't 
"off the cuff", and each had something new to say* 

These are people who are able and willing to say clearly something they feel 
strongly, even though they may be at variance with popular notions and current 
fads and fashions. This is a trait that is infrequent in professional education. 
We have a very curious notion of professional courtesy in education. It maJces it 
difficult at times for us to say tilings that someone will find objection to. If it is 
the "right" thing to say, we say it; if it is the "wrong" thing to say, we say noth- 
ing at all, forgetting completely that when the ostrich buries his head in the sand, 
not only does he present a ridiculous sight, but also a very inviting target. 

Another characteristic about all five presentations was that of "boldness". The 
questions you raised suggest that some of you may disagree, but the frankness of 
the papers permitted and invited this. 

Having read some of the papers before today, I was reminded of the preface to a 
delightful little book, Bergen Evans' Natural History of Nonsense , which some of 
you may know. It came out in 1946, and no^y it is available in a paperback edition. 
In his introduction, Bergen Evans said something which reminds me of our speak- 
ers. He said: "Until about one himdred years ago, rational men lived like spies 
in enemy country. They never walked abroad unless disguised in irony or allegory. 
To have revealed their true selves would have been fatal. Today, their status is 
more that of guerillas. They snipe from cover, ambush stragglers, harass re- 
treating rear guards, cut commxmications, and now and then execute swift forays 
against isolated units of the enemy. But they dare not yet risk an open engage- 
ment with the main force. They would be massacred. Their life is dangerous 
but exciting and is warmed by a sense of friendsmp not oft.en known Jimong the dull 
conscripts of orthodoxy. This book is intended e a sort of handbook for the young 
recruits in the gay cause of common sense. It indicates where the main armies of 
ignorance are now encamped and tells in a secret code what garrisons are imder- 
manned or mutinous. It tries to show the use of covering camouflage and techniques 
of infiltration and intrigue. It maps roadblocks and minefields and shows how to 
rig a booby trap. It warns of coxmter -espionage and it gives, again in code, the 
five infallible signs to know a foci. When the recruit has finished with it, he can 
toss it over the wall into the enemy's barracks. It may encourage desertion." 

I say very seriously that each of the persons today represents that kind of spirit. 

X must say one more thing. One note that has never been made at these conferences, 
either here at the regional one or at the national one, is somethiig that I thinkneeds 
noting. ETS is a private organization, but this is an organization that does better 
the very kind of a thing that a professional organization ought to be doing for itself. 
Here is an organization whose purpose is to develop tests, and it must exist on its 
ability to sell tests. On the other hand, it is not merely serving the profession in 
the commercial sense, but also is taking on a moral responsibility and obligation 
to educate and be provocative to the profession. 
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These sessions each year constitute a very important professional experience for 
people in education. I think it should be noted publicly that this is being done by an 
organization which is historically the kind of organization which doesn^t have this 
kind of altruism. I am very pleased to note the foregoing because I am not asked to 
nor am I padd to. 

Finally, I wish to make clear what you may have missed. This morning I engaged 
in some dueling with Mr. Rourke^ It wasn^t actxxally dueling, because I seemec^ to 
be on the wrong end of the sword. Some of you may have misunderstood this for a 
discourtesy on my part. Actually we do know each other and we are friends. I 
admire him greatly. I want to say right now that I think he is a very brilliant man. 
Indeed, if I were on tiial for murder, I can4 think of anyone I would rather have to 
defend me. The only trouble would be that he would have been th^ victir^ 

Again, I would like to extend my own personal appreciation and, I am sure, the 
appreciation of all of you to the people who spoke today: Dr. Tyler, Mr. Rourke, 
Dr. Davis, Dr. Kendrick, and Dr. Ebel; and also to the Educational Testing Serv- 
ice for what I sincerely consider to be a very valuable contribution to the profession 
of education. 

I think that there is no official benediction; Dr. Helmick refuses to come up before 
you again, so I will say good-bye on his personal behalf as well. 



(The meeting was adjourned at 3:30 p.m.) 
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